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ABSTRACT 

One of four programs funded nationally to test the 
notion that increased efficiency in the use of student and teacher 
time would result in higher student achievement, the 
Cotopaxi/Westclif f e Project was unique in that the two Colorado 
districts involved were small, rural, and operated on a four-day 
school week. The 4-year intervention consisted of full day, monthly 
staff development sessions for both teachers and administrator. 
Content was drawn from effective schools research and conventional 
wisdom about educational practice and was intended to improve student 
achievement by providing inservice to their teachers. Peer 
observations and teacher support groups facilitated the change 
process. A research component was in place throughout the project to 
monitor " time-on-task/engagement rates" and student achievement. 
Baseline data indicated quite high engagement rates during the life 
of the project. Achievement scores did not reflect this same degree 
of increase. This final report preser's: (1) a complete description 
of the project context; (2) content and process of the training 
provided; (3) effects of the inservice on teachers' behaviors and 
attitudes; and (4) changes in student engagement rate and 
achievement. Recommendations for other small rural school districts 
are offered on the basis of this effort. A technical appendix 
contains copies of the instruments used to collect data for the 
evaluat ion . ( Autho r/NEC ) 
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ABSTRACT 



The Cotopaxi/Westciiffe Follow Through Research Project was one of four 
programs funded nationally designed to test the notion that increased efficiency in 
the use of student and teacher time would result in higher student achievement. 
The Cotopaxi/Westciiffe Project was unique in that the two districts involved were 
small, rural and operate on a four-day school week. 

The four-year intervention consisted of full day, monthly, staff 
development sessions for both teachers and administrators, held on Friday, the 
"fifth day" when school was not in session. The content of this staff development 
was drawn from effective schools research and from the best of conventional 
wisdom about educational practice. Peer observations and teacher support groups 
facilitated the change process. A research component was in place throughout the 
life of the Project to monitor "time-on-task/engagement rates" and student 
achievement. 

Baseline data indicated that engagement rates were quite high at the 
beginning of the Project. Increases in engagement rates during the life of the 
Project were statistically significant. Achievement scores did not reflect this same 
degree of increase. 
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INTRODUCTION 



This is the final report of a four year Follow-Through Project involving the 
school districts of Cotopaxi & W^stcliff e, Colorado. This project was intended to 
improve student achievement by providing inservice to their teachers. An 
underlying assumption of this project was that this direct service to teachers would 
be translated into changes in student behavior. This layered pattern of impact is 
illustrated in Figure 1. The evaluation of this program examined each level to 
determine the impact. Change had to be demonstrated at each level before change 
at levels more distant from treatment could be attributed to the treatment. 



Figure 1 



Layered Pattern of Impact of Follow-Through Projects 

Level I - Change in knowledge of teachers and administrators 

The provision of information to teachers and administrators 
improves the knowledge they have about how children learn, 
what they need to learn, and effective instruction practices. The 
creation of climate in which change can occur facilitates the 
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Level II - Change in behaviors of teachers and administrators 

The application of the new information in the teacher's 
classrooms and in the administrators' activities will result in 
changes in use of time, in use of instruc tional techniques, in the 
support and collegiality of the teachers for each other and in the 
-^atmosphere of the schools generally. ^^^^^ 



Level III - Change in behaviors of students 

The improved techniques used by teachers in their classrooms 
will result in fewer discipline problems, higher engagement rates, 
.higher success rates, better motivation and lower absentee rates^ 



V, 



Level IV - Improvement in achievement test scores 

Assuming a good match exists between what is taught in the 
classroom and what is tested, the achievement test scores will 
improve. 



The report is organized in the following way. First a complete description 
of the project context is presented. It is our intention that readers will be able to 
determine the gcneralizability of the project's results by comparing their situation 
to the context in which this project took place. 

Second, a description of the training which was provided is included. Both 
the content and process which were presented arc described. In the third chapter, 
the effects of the inservice on teachers* behaviors and attitudes are discussed. The 
fourth chapter contains a report of the changes in student engagement rate and 
achievement. A technical appendix is included which contains a discussion of the 
instruments used to collect data for the evaluation. 

Each section of the report is intended to be independent of the other 
sections, although all the sections are required to obtain a comprehensive 
understanding of the project. Some information is presented more than once and 
will seem redundant to those readers who read the entire evaluation. When 
possible, readers are referred to appropriate sections of the project for more 
detailed information. 
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INTRODUCTION 

In July of 1981, the Cotopaxi/Westcliffe Follow Through Project was one of four 
contracts negotiated with the National Institute of Education to explore the viability of 
improving student achievement in mathematics and reading, primarily by increasing the 
efficient use of student and teacher instructional time (time-on-task/engagement rate). 
The other three projects involved the school districts of Detroit, Oakland and Napa 
Valley. The four year contract called for the development of an intervention strategy for 
bringing about these increased time efficiencies and a research design capable of tracking 
the impact of the intervention. 

The Cotopaxi/Westcliffe Follow Through Project was unique in a number of 
different ways. 

1. It was the only project to involve small rural schools. 

2. The two participating schools operate on a four-day school week, an 
option provided by the state of Colorado to help districts cope with 
energy costs and budget restrictions. 

3. The Project was a collaborative effort involving two school districts, an 
institution of higher education and a regional educational laboratory. 

THE CREATION 

The initiative to compete for the Follow Through contract originated with the 
Mid-continent Regional Educational Laboratory, (McREL). The RFP was attractive for 
two reasons. First, it specified that it wanted to target a rural school population as one of 
the programs to be funded. The Laboratory had an ongoing Rural Education Project with 
a mission to address the unique problems of small rural schools. Secondly, the purpose of 
the RFP, the raising of student achievement through the more efficient use of time, 
dovetailed nicely with another of McREL's program components designed to translate the 
emerging "effective schools" research into a school improvement program. Furthermore, 
Colorado had just recently changed the definition of the school year which allowed 
schools to operate on a four-day school week. If it was possible to find interested schools 
that were on the four-day school week, it might just be the added dimension to catch the 
interest of those making the funding decisions. Getting this interest and commitment at 
the local district level was also necessary to meet the RFP requirements that the 
contracting agency be a local education agency. 

Knowing that small rural school districts have neither the expertise nor the time to 
develop competitive proposals, the Director of McREL's Rural Education Project visited 
with staff from Colorado State University who had been conducting a study of districts 
implementing the four-day school week. If a couple of these schools could be persuaded 
to participate in the Follow Through program, the study could serve a dual purpose. 
Because of the extensive data collection which would be required to monitor effective use 
of time, a proposal could be written that would not .only address the "more efficient use 
of time increased/student achievement" issue, but could enlighten some questions 
concerning the viability of the four-day week as well. One such issue, for instance, was 



concerned with whether or not early elementary students, K-4, could learn as well in a 
four-day week with its 20% longer school day as they do in the traditional five-day week. 
The CSU staff agreed that there were some possibilities worth pursuing and arranged a 
meeting with the superintendents of the two neighboring schools of Cotopaxi and 
Westcliffe. 

At this meeting, the content of the RFP was discussed along with the possibility of 
developing a collaborative effort involving the two districts, Colorado State University 
and McREL. Since the RFP required two sets of K-4 classrooms, both districts would 
need to be involved. Furthermore, one of the districts would need to serve as the 
contracting agency with the superintendent being designated as "principal investigator." 
The superintendents agreed to discuss the possible participation of their districts with 
their school boards and, in so far as possible, get reactions from those teachers that would 
be involved. School had been dismissed for the year and many of the teachers were gone 
for the summer. 

Staff members from Colorado State University indicated an interest in assisting 
with the delivery of the necessary staff development if the project was funded. McREL 
agreed to develop the proposal and, if successful in the competition, conduct the research 
component. 

The contract was successfully negotiated and the consortium involving the two 
^school districts, the University and the Laboratory continued for the duration of the 
Project. The arrangement had both its strengths and weaknesses as win be discussed later 
in the report. 

THE SETTING 

The neighboring school districts of Cotopaxi (Fremont County RE 1) and 
Westcliffe (Custer County C-l) lie along the east slope of the Sangre de Cristo Mountains 
in South-central Colorado. The Cotopaxi District covers 540 square miles dominated by 
the Rio Grande River and the mainline of the Rio Grande Railroad. The business 
district, clustered along Highway 50, consists of a general store, filling station, a cafe and 
a small pottery business. The river rafting expeditions, which begin in the lower end of 
the district at Texas Creek, represent a major tourist attraction. 

The Cotopaxi Schools sit on a man-made bluff carved out of the mountains on the 
north side of town. The buildings are a mix of an old two story building, constructed in 
the early 30's, and newer classroom wings for the elementary grades and the specialized 
facilities required at the secondary level. The majority of the teachers commute from 
their homes in the larger neighboring communities some 45 minutes away. Ninety-four 
percent of the 243 students are bused. 

Twenty-six miles to the south, via a winding road that connects the Rio Grande 
Valley with the Wet River Valley is the town of Westcliffe. Because of its location, 
Westcliffe is much more of a self-contained community, providing a good variety of 
services for a town of approximately 400. Ranching is still the mainstay of the economy, 
supplemented by a number of summer camp operations along the Front Range to the east 
and a developing ski area to the west. 

The Westcliffe schools are of more recent vintage, being of one design, with the 
elementary classrooms on one side and the secondary on the other. Separating the two arc 
the administrative offices and the gymnasium. The teachers live in the community or in 
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the surrounding area. The 700 square mile district enrolled 393 students K-12 when the 
project began, 60% of whom were bused. 

Both of the elementary schools participating in the project would be considered to 
be traditional, e.g. self-contained classrooms, one teacher per grade level, with relatively 
small classes. Administrators and support personnel such as principals and counselors are 
responsible for the entire K-12 system in each of the two districts. In Cotopaxi, 53% of 
the students participated in the free or reduced price iunch program; in Westcliffe, 33% 
of students participated with another 17% eligible. Both schools qualify for Title I; 
Cotopaxi has a 3.6% minority population (i.e., 8 students), Westcliff?., 6.1% (18 students). 
The school is the largest employer in both communities where the average income is 
$10,000 per family. 

When the Project began, Cotopaxi's reading scores on the Iowa Test of Basic Skills 
indicated 66% at or above the norm; 44% of the K-4 students were at or above norm in 
mathematics. Test scores were not available for all grades in Westcliffe, however, 
information available for grades 3 and 4 on the Iowa Test of Basic Skills indicated norms 
approximately one year above grade level in both reading and math. 

THE FOLLOW THROUGH PROJECT; AN OVERVIEW 

The general parameters of the Cofopaxi/Westcliffe Follow Through Project were 
consistent with those laid out in the RFP. All students in grades K-4 in the two schools 
would be involved. The emphasis was to be on increasing achievement in reading and 
mathematics through the more efficient use of student and teacher time. A research 
component would be developed to monitor the changes in the use of time (engagement 
rates), and any impact on student achievement. 

The proposed intervention to increase engagement rates was a series of full day 
workshops (approximately one each month during the school year for the life of the 
project) to be held on Friday, the "5th day" when school was not in session. The schools 
would alternate hosting the sessions. Consultants from Colorado State University and 
McREL, provided by the Project, would be responsible for assisting in the planning and 
delivery of the staff development activities. The Project would also pay the participating 
teachers and administrators an extra day's salary for attending the in-service sessions 
since they put in the equivalent of a regular week's work in four days 'vith the 
alternative schedule's, 20% longer school day. 

The general routine called ior the consultants to travel to the site on Thursday 
morning, approximately a four hour drive, visit/work with teachers and or administrators 
in their buildings on Thursday afternoon. Teachers and administrators would then both 
participate in the day-long work session on Friday, Staff members were left with tasks or 
practices to work on during the month. During the next session, the successes snd 
problems encountered during the month would be discussed prior to moving on to ne* 
activities. The cycle would then be repeated. 
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GETTING STARTED 

Durine preliminary planning sessions, concerning how best to get the Project 
underway the decision was made to focus on the reading and math curriculum. Neither 
of the schools had engaged in any recent curriculum work nor did they have written 
curriculum guides. It was argued, that if increasing engagement rates was to have any 
impact on studeai achievement, the curriculum needed to be sufficiently coherent so that 
if mastered, it would result in higher achievement. A series of Friday work sessions were 
devoted to structured "curriculum mapping" activities to insure that content was being 
covered in a sequential and articulate fashion. 

During the months that the curriculum work was taking place, classroom observers 
were being trained and the baseline data collected on engagement rates. Since one of the 
important questions to be investigated by the Project was ;ht ability of the students to 
make good use of the 20% longer school day, observations were made and data collected 
for the full day, not just during reading and mathematics. An analysis of the baseline 
data indicated relatively high engagement rates, 82.53% in math and 81.03% in reading. 

PROJECT PLANNING 

During the life of the Cotopaxi/Westcliffe Follow Through Project the planning 
process was characterized by parallel activities at two levels. One level was concerned 
with the longer range planning of Project activities. These sessions always involved the 
superintendents and the consultants, and depending on the issues and time availability, 
the principals and teachers as well. (A number of such planning sessions are of note 
and are discussed below.) The second level of planning tended to be ongoing and was 
concerned more with the fine tuning of Project activities. The last portion of each 
inservice day involved the participants in an informal evaluation activity, the results of 
which were discussed by the consultants on the drive back to Denver and used to help 
shape the next inservice session. 

MAJOR PLANNING SESSIONS 

The first major planning session, once the Project was underway, took place when 
the participants were presented with baseline data on engagement rates and achievement 
scores in reading and mathematics. The engagement rates appeared to everyone to be 
quite high and no one was particularly concerned about the level of student achievement. 
A lengthy discussion followed concerning whether or not the Project should pursue a 
narrow sharply focused effort to attempt to drive engagement rates even higher, QL 
develop a more balanced set of activities which would expand the teacher's repertoire of 
instructional strategies, always keeping in mind the bottom line of the Project, e.g. how do 
these strategies contribute to or stand in the way of more effective use of student and 
teacher time. 

A persuasive case was eventually made for the latter option, based on the fact that 
the curriculum and instruction in small rural schools is typically quite limited, and fairly 
basic It was agreed that finding ways to enrich the learning environment might be as 
important to the improvement of student achievement as increasing the efficiency of that 
learning environment. As a result, the Project attempted to put together a parallel, but 
integrated, staff development program which included various models of teacher ana 
"effective schools" practice. 

A related set of issues concerned with staffing the inservice also came under 
discussion during the same meeting. The original plan had called for using a variety of 

ERIC / 10 



different consultants, drawing both on the resources of Colorado State University and 
McREL. Feedback from the teachers and administrators suggested that it would be better 
to find two trainers that could address the broader focus discussed above (effective 
schools practices and expanding the repertoire of teaching strategies), and stay with those 
individuals, thus providing consistency and building the trust that results from an 
ongoing relationship. 

The planning session at the end of the second year of the Project was facilitated 
by a computer driven decision making process which allowed all staff members to be 
involved in establishing program priorities for the following year's activities. A two day 
planning retreat was held prior to the last year's activities for the district administrators, 
consultants and teacher representatives to reflect on activities to date, review the proposal 
to see if the Project was doing everything it said it would do and structure the final 
year's program. 

PROGRAM COSTS 

The total cost of the four year Cotopaxi/Westcliffe Follow Through Project was 
just under $240,000, or approximately $60,000 per year. Of the $240,000, approximately 
two-thirds of the costs were related to the research, reporting, and administrative 
requirements of the RFP which included periodic travel to Washington, D.C. 
Approximately $80,000, or $20,000, per year was spent for the actual staff development 
interventions to improve the effective use of time. 

In developing the Project, careful consideration was given to designing a project 
which would not be so expensive that other districts could not consider implementing a 
similar effort. Cost considerations contributed significantly to the decision to go with 
part-time personnel from the University and the Lab to carry out the various functions 
rather than hiring full-time personnel. Lab personnel were contracted on a daily rate to 
develop and conduct the research component and, for all practical purposes, carry out the 
duties of the principal investigator. Staff members from Colorado State University were 
contracted for in the same way for staff development work. 

The $20,000 per year costs for a comparable staff development program, e.g. two 
consultants for two days/month during the school term, plus paying the teachers and 
administrators for the extra day would not be out of reason, particularly if two or more 
districts joined together in a cooperative effort. An analysis of the cost savings realized 
in moving from a 5 to 4-day school week are sufficient to cover the costs of such a school 
improvement program. As we chall show later in the project results, elementary students 
appear not to be adversely affected by the 4-day school week. This in effect, then, gives 
local board members a choice of using the education dollar to bus students 5 days a week, 
or go to a 4-day schedule and invest that money in instructional improvement. 

PERSONNEL CHANCES DURING THE LIFE OF THE PROJECT 

Conceptualizing a field based research proje;t is one thing. How things actually 
get played out over time is generally something quite different. A number of personnel 
changes took place, both in the schools and in the part-time project staff, which impacted 
on the operation of the project. These will be discussed as a way of providing additional 
background and understanding before moving on to Project outcomes and lessons learned. 

While maintaining job tenure may or may not be any more difficult in small 
schools than in large schools, changes in staff tend to have a greater impact in small 
schools simply because there are so few of them and the relationships tend to be much 
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more personal than in the large school setting. The superintendent as well as the 
principal, for instance, work directly with the classroom teachers. When a third grade 
teacher leaves, the whole third grade staff turns over. There is not the bureaucratic 
inertia which comes from numbers to keep programs in place. 

The Cotopaxi/Westcliffe Project experienced a number of staff changes during the 
four years, some of which had more impact than others. At the end of the first year of 
the project the Cotopaxi superintendent, who had been designated principal investigator, 
resigned. In so far as can be determined, the problems with the board, which precipitated 
the resignation, had nothing to do with the Project. According to the superintendent, 
there were personality differences with members of the board and a disagreement on how 
best to deal with re-sodding the football field. The end result, however, was the need to 
designate a new principal investigator and negotiate these changes with NIE. Rather than 
leaving il.vSe duties, which primarily involved overseeing the financial aspects of the 
Project, with a new superintendent, the decision was made to move this responsibility to 
the superintendent in Westcliffe. 

The principal of Cotopaxi, who had been a part of the Project from the beginning, 
was appointed superintendent which helped to provide continuity in the project. Th'e 
person filling the principalship was new to the system and had to be oriented to the 
project activities. These were the only staff changes that took place in Cotopaxi. 

The Westcliffe situation remained stable until the fourth year of the project. 
While one new teacher was brought in during the second year of the project because of 
large enrollments at the first grade, there were no resignations until just prior to the final 
year. At that point, the principal, who had become one of the strongest leaders in the 
program, left for another principalship; his wife, one of the project teachers, went wit^ 
him. Two other teachers also resigned, one to join her husband in another location; the 1 
other to take an assignment Lj another state. Losing 4 out of a staff of 7, with one being 
the principal, left a sizeable hole in the program. In order to bring the new staff along as 
quickly as possible one of the strong teacher-leaders was released part time to serve as a 
resource person. There was, however, a significant loss of momentum because of these^ 
changes and no doubt some deterioration in overall project impact. 

There were also some important changes with the support personnel at McREL 
which impacted the program. The person responsible for developing the original research 
design left the Laboratory before the contract was negotiated. His replacement at the lab 
rewrote the design which was incorporated in the program, but because of other interests, 
never invested the necessary time to see that the program was well implemented. It 
finally took the interest and skills of a research assistant to get the research component 
on track. And, while all these difficulties did not greatly impact the overall effectiveness t 
of the research component, e.g. the necessary data was collected as well as possible with 
the existing state-of-the-art, and analyzed for project impact. It did not however, 
function in such a way as to provide the periodic feedback to teachers and administrators 
which had been hoped for and which may have contributed to the overall effectiveness of 
the staff development effort. 
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INTRODUCTION 

As mentioned in Chapter I, the project as conceptualized and the project as 
conducted are two different entities The treatment in this project consisted of inseryice 
provided to teachers and included an eclectic, sometimes " seemingly incompatible, 
collection of techniques, procedures, and methods. The inclusion of these diverse 
technologies was a deliberate training decision. The outcomes of the project are bet 
understood in light of the training which was provided. This chapter contains a 
description of the content and the training process. 

CONTENT OF THE STAFF TRAINING 

The staff development intervention was, as indicated earlier, shaped in part by the 
participants themselves. The guiding rationale was: "What works and how can it be 
presented and applied in an effective and affordable manner?" Efforts were made to 
ground the activities in recent research or the best of conventional wisdom about good 
' practice. The substance of the inservice was drawn from a number of sources. The first 
source was the "effective schnnls research" , e.g. effective use of time/increased 
engagement rates, creating an orderly climate for learning, developing strong leadership 
high expectations for students, focus on the basics, (reading and mathematics) careful 
monitoring and feedback of student progress. The time issues were addressed in two 
ways The curriculum mapping detailed the amount of time each project teacher spent on 
reading and math on a regular basis. To increase engagement rates, teachers were exposed 
to a variety of ideas and techniques intended to reduce classroom disruptions, to decrease 
the time needed for transitions from one activity to another and in general to improve the 
climate for learning. 




A third dimension of the trainHg program was based on the developmental notions 
of Piaget and Bruner. Teachers were assisted in assessing the cognitive levels at which 
students were functioning, and how to better challenge them actively in the earning 
process, thus stimulating their cognitive growth. The final dimension of the substance of 
the training program was to broaden the variety of instructional approaches available in 
the classroom, Pnn perative learning and peer tutpring were introduced to increase the 
involvement of students in the learning process, provide active assistance and more 
immediate feedback for each other. Concept attainment, inductive th.nk.ng . injuun. 
l earning svnectics . a technique for fostering creativity, and storv telling were introduced 
to increase both the activity level of students and the quality of their engagement in 
learning. 

The introduction of challenging student-centered instruction created more group 
and interactive experience, more focus on problem solving, more opportunities ior the 
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teachers to play active roles as facilitators for learning. The decisions to include such 
activities were made consciously, recognizing that by so doing the high engagement rates, 
attributed to a generally orderly climate and the extensive use of seatwork, might be put 
at risk, at least temporarily. 

STAFF TRAINING PROCESS 

From the Project's very beginning, attention to the process of training was given 
consideration equal to that of the products or content of training. The full day, monthly 
inservice sessions appeared to meet the findings of the 1978 Rand study regarding the 
need to provide adequate time and follow-up for innovations to be successfully adapted 
and adopted locally. In addition, staff members from the two project schools were 
actively involved in the planning for and delivery of training. One time appearances by 
outside experts were avoided. Other teachers were utilized to demonstrate successful 
techniques both via videotape and directly in the project school classrooms. 

A peer coaching model involving scheduled peer observations and regular support 
group meetings, based on the work of Joyce (1982), was introduced to provide an ongoing, 
practical and cost-effective vehicle for the implementation of new practice and solving 
the problems which emerge as the result of this implementation. Finally, careful attention 
was given to the general learning climate of the schools, Sarason (1980) and Lezotte, et al. 
(1980) as a way of fostering increased student learning. 

In the following portion of this chapter, one trainer describes the training which 
was provided to meet two of the project objectives: 

1. To create an environment in which "change" is fostered. 

2. To improve classroom instruction through the use of research-based 
classroom practice. 

CREATING A CHANGE ENVIRONMENT FOR IMPROVING CLASSROOM 
INSTRUCTION 

Creating an environment for change was no accident with this project. A major 
portion of the resources, both time and money, was dedicated to this end. The model 
conceived for use in the Project was unique and thus bears close examination. 

There is a specific set of assumptions underlying the creation of a change 
environment that were drawn upon in developing such a climate in the 
Westcliffe/Coiopaxi Project. Before we actually describe the model developed for use in 
the Project, it is important to understand these assumptions and their application in a 
school setting. 

Assumption 1: We are motivated to chanee when we believe that a pa rticular 
activity or practice will satisfy a need or desire we have . The emphasis here is on the 
need as identified and defined by the individual rather than the need expressed by a 
designated leader or group. It may be true, however, that as a result of a suggestion by 
another, we are able to see our individual need being met and, thus, will change. The 
decision to change, however, is still with the individual. 

This is particularly germain to 'he school setting wherein the principal often is the 
. person expected to make suggestions tj a teacher for change in behavior. Often, however, 

8 

ERIC - , id 



the desired change does not occur as a result of these suggestions. If change does occur, it 
is often temporary and once accountability is relaxed it disappears. The reason is that the 
need is one seen by the administrator rather than by the individual expected to make the 
change-the teacher. Thus, an important concept to this particular change model is that 
the individual teacher is the person who is expected to identify the change and mane the 
appropriate decisions with respect to modified actions. 

Assumption 2- Frrrlhnrk ig necgjfao change tn occur. Feedback has two 
functions. It provides a starting point for change and it allows for adjustments in 
behavior to meet the desired outcomes as the process of change takes place. This 
feedback is required by both the individual and by a "group" for behavior to change. 

Feedback in many schools simply means a periodic observation and review (or no 
review at all) of a teacher's performance. "Periodic" might be defined from once to 
perhaps a maximum of three or four times a year. This is usually done by the principal 
or his/her designee. This type of feedback does not meet the conditions required for 
change in teacher behavior to occur. First, the "starting point" for the change is often too 
late in coming. Second, behavior can hardly be adjusted when feedback is provided sp 
infrequently. 

Assumption 3: Sunnort is an essential e lement of change. The more support we 
receive from others for our actions, the more likely it is that those actions will continue 
" and strengthen. This concept has been proven many times with such movements as Weight 
Watchers, Smoke Enders, AA and the like. The support must be consistently present over 
time in order to make a difference and be provided by others who are able to empathize 
because of similar needs or experience. 

Assumption 4: The primary "experts" in classroom instruction within a school are 
the professional staff. 95 percent of whom are classroom teachers. Thus, the experts arc 
the teachers. An adjunct to this assumption is that, generally speaking, teachers believe 
each other in terms of instruction more than they do other roles in the educational 
setting, i.e., principal, professor, central office personnel, etc. 

Assumption 5: There is a known body of characteristics that the research tells us 
are more effective than others in increasing student achievement and performance; 
certain specific classroom p ractices work better than other?. The existence of these 
practices can be observed in the classroom and, further, these practices can be learned by 
teachers and implemented in their classrooms. 

This body of research-based teaching characteristics provides the basis for the 
improvement ef'-Tts in the school. While the "change environment" is the "process 
vehicle for imprc emcnt in the school, these research-based practices are the content for 
increased effectiveness. 

Assumption 6: The primary educational leader in the sch ool is the principal . 
There are specific leadership behaviors that work better than others in facilitating a 
program of instructional improvement. 

These leadership behaviors can be assessed in each principal and a planned 
program of leadership development can be implemented to assist in the school s overall 
improvement efforts. 
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PEER-CENTERED COACHING MODEL: A VEHICLE FOR CLASSROOM CHANGE - 

The model utilized by the Project staff to impiement change and thus improve 
instruction in the classrooms of the two schools came to be known as the Peer-Centered 
Coaching Model. It encompassed the following elements: | 

- motivation to change aimed at satisfying individual needs, as well as A\) ^ 
group identified needs; > ' 

feedback for teacher and administrator behavior; 

focused support, teacher-to-teacher and group-to-teacher; 

drawing upon existing instructional "expertise"; 

- creating a new instructional expertise through sharing of research-based 
classroom practice; 

- examining and improving leadership practicss by the school 
administrators. 

Many school programs suggest that some or all of these elements of improvement 
" and change occur incidently (or in some cases, accidentally). The major difference with 
this NIE project is that ALL of _t he_sf, elements .wcre-prcsmt Jhroughout the program and 
wcreTQacL.ituictiired into a specififi_model f rom the.Jb.c ginning Th e particular model 
of coaching developed for use in this Project was based upon work from a number of 
various researchers, practitioners and disciplines. We will now examine this model and its 
specific characteristics. 

The Peer-Centered Model has two major components, namely: the use of "peer 
observations" and implementation of a support system. Both components are vital to the 
maximum success of the model as a vehicle for change and improvement. A description 
of each component follows based on the experience obtained with the staffs of the Project 
schools. 

PEER OBSERVATION 

The general guidelines for the "peer observation" were as follows: 

1. Team up with somebody you like and/or respect. The process depends 
upon credibility and trust. 

2. Maintain a non-judgmental, non-evaluative posture when giving and 
receiving feedback. Remember: We only make change when we are 
motivated by some need or desire WE have. 

3. Use the opportunity to develop a "trust partner"— one with which you 
can feel free to brainstorm new ideas, challenge old ones, reveal 
frustrations and failures, as well as successes, and generally be more 
vulnerable than, perhaps, you feel you usually are able. 

4. Expect support and encouragement from your principal. Seek his/her 
help in setting up your peer observation schedule. Avail yourself of 
his/her expertise, when appropriate. Remember: Your principal 
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supports this program and has expressed his/her availability to help you 
to make it work. 

The process began at the outset of the second year of the Project. At this point, 
many of the resource materials used to support the Coaching process were not developed. 
Further elaboration of the model and how it was implemented are described below: 

Teachers form groups of two or three. While the two-person team seems to 
work best, there may be situations where three teachers are able to more 
easily schedule their observations. This team will remain as a group for at 
least the first six to eight observations. After that, teachers may choose to 
team up for the Peer Observations on an ad hoc basis, dependent upon the 
specific practices to be observed, the availability of the observation partner, 
etc. Teachers often form their groups on the basis of similar grade levels or 
subject areas, but this is not necessary. 

In one of the Project schools, an interesting phenomenon took place. A number of 
Project teachers actually tried teaming up with secondary teachers who are housed in the 
same building. The success of this approach was neve- examined in detail, but did seem 
to be satisfying for a few of the teachers. The essential ingredient in the Peer 
Observation process is the giving and receiving of specific feedback in a safe and trusting 
environment and thus the grade level expertise of the observer is not crucial to the 
process. 

The two teachers then developed an observation schedule with the assistance of the 
principal Because the Westcliffe and Cotopaxi school districts were quite small the 
superintendents of each of the sites also were able to provide assistance. The length of 
time for each observation varied dependent upon the specific practices being observed. 
For example, a teacher may ask his/her peer observation team member to record how 
quickly the students in a particular Reading group begin to work on an assigned task 
when the teacher moves to a second group. This might require the observer s presence in 
the classroom for only a few minutes to observe that specific activity. In another 
instance a teacher may want to find out if he or she is allowing opportunities for all 
students'to answer teacher-directed questions. This may require an observer being present 
for a longer period of time. The length of time spent in the classroom by the observer 
varied considerably from classroom to classroom, as well as from school to school because 
of the procedures followed in each of the schools to "free up" teachers to do the 
observations. 

In one of the schools, it became apparent from the beginning that teachers had 
very little non-assigned time to be available for such observations. While various plans 
were attempted such as the principal taking over classes, the use of parent volunteers and 
the like the decision was finally made to pay a substitute teacher to come into the school 
one day per week, twice a month, for the express purpose of covering classes. This 
enabled teachers to be free to observe in a colleague's classroom. The principal also 
covered some of the classes, as well. 

In the other school, the "coverage" for teachers observing was handled primarily 
through the use of "specialists" such as music and art teachers. The observations took 
place when classroom teachers were freed by the presence of these specialists in their 
classrooms. In addition, the principal, and in some instances, the superintendent covered 
classes. Some parent volunteers were also used. 
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The team members met prior to the actual observation to determine who would 
observe who, and specifically what practices were to be observed. The first observations 
were made with little or no focus on specific practices in order to allow the participants 
to become more comfortable with having another teacher in their room. Of course, it is 
the observation and feedback of specific classroom practices that is key to the growth and 
development of the teachers involved. As the training process continued, specific 
classroom practices associated with the instructional effectiveness research were presented 
by the inservice consultants at each of the subsequent Friday staff development days. 
These research-based practices, which are tied to increased student performance, in fact, 
provided the basic content for the peer observations. 

In most instances, it was discovered that the optimum location for the observer is 
at the back of the room. This did vary, obviously, dependent upon the type of activity 
being observed. Other information, such as how long the observation should be, what 
type of lesson will be going on, etc. needs to be discussed in advance of the actual 
observation. A "Peer Observation Planning Sheet" (see Technical Appendix A) was 
developed and used for this purpose. 

Following the observation, the two teachers met to share what was observed. This 
meeting usually took place immediately following the observation session. In a small 
school such as those involved in the Project, it was fairly easy to adjust the time required 
to do both the observation and the follow-up conference. Usually this meeting required 
*~ very little time since it is essentially a feedback session. The average length of time for 
this session by the Project teachers was about ten minutes, although some teams used this 
-sharing" time for more in-depth discussion, which obviously required more time. The 
most important factor seemed to be that the meeting take place as soon as possible 
following the observation. A typical feedback session is as follows: 

a. The observer objectively reports on what he/she observed, stating what 
was observed as specifically as possible, quantifying the behaviors, 
wherever applicable, i.e., "When you moved to the second Reading 
group, I observed that three of the eight students appeared to be on task 
within the first two minutes. The others in the group appeared to all 
be on task within the next minute with the exception of one student 
who really never seemed to get going on the assignment." or, "I noticed 
that when you asked the question about the causes of the Civil War, of 
the whole class, five students immediately raised their hands. You 
called on one of them just as ten or twelve other students began to 
respond." 

b. The teacher then does a self-critique with respect to the specific t 
behaviors that were reported by the observer. For example, "I might 

have remained with the first Reading group for a minute or two after 
giving instructions on the assignment instead of moving on so quickly 
to the second group" or, "What I think I will try next time I ask a 
question of the class is not to respond to volunteers, but rather call 
upon students at random. This will allow every student an equal 
opportunity to answer the question." 

c. The two teachers share any experiences, ideas or techniques that might 
be tried to bring about the desired change. Again, NO judgments. 
Avoid "You should....Try saying, "I tried..." or, "I think this might work 
for ME in a similar situation." 
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THE SUPPORT GROUPS 



The second component of the Peer-Centered Coaching Model is the formation and 
implementation of the Support Groups. Although both school staffs used the two 
components of the Coaching model throughout the duration of the Project— peer 
observations and support group— the Support Group meetings were considered the "most 
useful" by the members of both school staffs. This is partly understandable because of 
the "social" nature of such meetings. It is further evidenced, however, that this medium 
provided a real learning experience in terms of improving specific classroom practices. 

The primary purpose of the Support Group in the context of the Peer-Centered 
Coaching Model is to provide a vehicle of support to help teachers as they continue to 
improve their instructional effectiveness. Essentially groups are comprised of teachers 
who have a common desire to improve their classroom effectiveness, as well as the desire 
to give and receive feedback. The Support Groups at both of the Project schools were 
comprised of four Peer Observation teams. For purposes of interactive discussion and 
participation, this appears to be the ideal number of group members. These groups were 
maintained as integral units throughout the first three years. During the final year of the 
Project, staff changes at one of the schools caused the original support group to totally 
reorganize itself. It was obvious when this occurred that the school which had retained 
its original group membership from the beginning of the program continued to operate 
smoothly during that final year. The Support Group in the other school, on the other 
hand, had to begin many of the support practices over again. 

It was hoped that the Support Group would be viewed by the teachers as a useful 
vehicle for professional growth, rather than as another way to take up their time; to add 
to their already over-burdened schedules. To avoid "runaway" sessions and keep the 
meeting on task, certain specific roles were taken by various members of the Group 
during the support meeting. 

A FACILITATOR conducts the business of the meeting. In fact, all of the group 
members assisted the process of the movement of the meeting by acting as facilitators for 
the group process. Part of the Peer-Centered Model training that took place on the Friday 
inservicc sessions dealt directly with this skill of group facilitation. While in the 
beginning, various group members volunteered to take this role on a rotating basis, as the 
groups progressed one person emerged as the facilitator and maintained this role for a 
period of time. 

The Facilitator was responsible for starting the meeting on time and ending the 
meeting on time, as well as keeping the meeting on task. The issues of staying on task 
and being "on time" are, perhaps, the two factors that contributed most to the breakdown t 
of meetings. When an individual is concerned about what he/she "ought to be doing" 
instead of what is going on in the meeting, that person is not a contributing member of 
the group. The times of the meetings in the Project schools were not extended unless 
arrangements had been made prior to the meeting date. 

The GROUP RECORDER was responsible for keeping a "group memory" of 
pertinent content of the meeting. Unlike the "minutes" in a traditional meeting, the group 
memory is taken on large sheets of newsprint and is recorded with a large marker or 
crayon so that all can view the "memory" as it is being written. The Group Recorder 
simply took notes of the important issues discussed at the meeting. When, during the 
course of a support group meeting, any Group member differed with what was being 
recorded, changes to the notes were made immediately so that all could see it can be 
changed. The newsprint "memory" was brought to each Group meeting and displayed for 




all to sec. The Group determined norms for this "memory" during the course of the first 
few meetings. The amount of detail of this "memory" varied with the two groups. 

As in the case of the Facilitator, the Group Recorder role rotated among the Group 
members in the beginning. As the process continued, one person emerged as the 
"Recorder." During the actual Support Group meeting, the Facilitator and the Group 
Recorder often switched roles when one or the other became so intensely involved with 
the discussion that he/she was not able to perform the respective role adequately. 

The four basic phases of the Support Group meeting formAt are: 

1. Development of the agenda 

2. Review and feedback about the previous meeting. 

3. Problem-Solving 

4. Closure 

Each of these phases or steps in the Support Group process began to be clearly 
defined as the process progressed during the first year of the program. Each of the 
phases is described in terms of the specific process that emerged during that first year 
and continued to be used throughout the remainder of the Project duration. 

DEVELOPMENT OF THE AGENDA. Once the meeting has been called to order, 
*~ the Facilitator requests agenda items from the members of the Group. The items are 
usually in the form of some problem or issue being faced by the teacher in the classroom. 
Since the peer-centered Coaching Model's objective is "classroom instructional 
effectiveness," the issues discussed during the Support Group meeting tend to be 
classroom oriented. Usually the problem or issue is related to a technique or practice that 
a teacher has been trying to improve and perhaps has been the focus of observations with 
his/her Peer Observation team member. This "content" usually came as a result of a 
technique or practice which was presented as part of the training provided to teachers at 
a previous Friday training session. 

The agenda item is stated in simple, behavioraTterms. No elaboration is necessary 
at this time. The teacher providing the agenda item also estimates the amount of time 
he/she thinks will be required by the Group to "handle" the problem. For example, "I 
know that when I move from one Reading group to the second, the first group does not 
seem to get working on the assigned task immediately. I'm not sure what I might do so 
that they get started working right away. I think this will take about 15 minutes of the 
Group's time." Since most Jiupport Group meetings are no longer than forty-five minutes 
in length, only one or two problems or issues will be handled. The purpose for requesting 
a certain amount of time is that this helps the Group to establish some Group norms 
around time. It is not expected that by stating an estimated time that this must be "held 
to" during discussion of that item. Again, rather, this is a way for the Group to begin to 
establish some norms for itself. The Group Recorder lists ;he possible agenda items as 
stated, in note form, with the amount of time requested and the person's name. 

After all the items have been posted on the group memory, the Facilitator asks the 
Group to prioritize the proposed items to determine which items will be discussed at this 
meeting. In almost no case will there be enough time during a single meeting to discuss 
all of the agenda items. The purpose of the prioritizing session is to try to help the 
teachers who are most in need of help at this meeting. During the Friday inservice 
training sessions, the Project teachers practiced techniques for prioritizing and for 
determining who had the greatest need. Such techniques as "asking for what ycu want," 
using "I" statements, etc. were practiced in the training. For example, "I would like help 



from the Group during this meeting because I am meeting with the parents of this student 
o d™ cuss this issue nfxt week and so I need to get some help nowr It s important : that 
even though an issue may not be handled by the Group because 01 men ot time or »ow 
priority, the person providing the issue was "heard" In many instances, members whose 
problems were not handled at the meeting found that other Group members offered their 
support and assistance following the meeting time. The process of agenda development 
allows each member an opportunity to ask for help from the group and to create an 
five" men, ^ TtnT agenda hems. The entire process, after practice, usually requires no 
more than five minutes. Items that are not handled by the Group during this meting are 
not placed automatically on the next agenda. If the issue is still considered '"iportant ^by 
a member and wasn't handled at this meeting, the teacher must again go through the same 
process at the next meeting. 

REVIEW AND FEEDBACK. The Facilitator draws the group's attention to the 
group memory from the previous meeting. The teacher(s) who was the object of the 
previous meeting's problem-solving activity reviews what actions he/she has taken to 
remedy the problem and what progress has been made since the last meeting. (Dying the 
"Closuie" portion of the previous meeting, the teacher(s) whose problem or issue was 
discussed by the Group made a verbal commitment to the Group to try some action and 
report back to the Group-See "Closure.") For example, "I tried remaining with the first 
Reading Group for a few minutes after I gave them their assignment to make sure they 
were all "on task" before moving onto the second group. All but two of that reading 
' group is now finishing their assignment by the end of the class period, so it is getting 
better than it was." 

The teacher may seek additional feedback from the Group at this time or the 
Group may ask the teacher clarification questions about the actions the teacher has been 
-taking If the teacher feels that no progress toward solving the problem has been noted 
after he/she tried some new actions, a request for the item to be placed on the agenda 
again would have been made during the "agenda development" portion of the meeting. 

In the beginning, the Review and Feedback portion of the meeting took on the 
flavor of a "story-telling" session and required considerable time to complete. <~a relul 
monitoring of this part of the meeting during the Friday insemce sessions reduced this 
time dramatically. After the first few meetings, this part of the meeting took less than 
ten minutes to accomplish. 

PROBLEM-SOLVING. The Facilitator calls upon the teacher whose agenda item 
was listed as the first priority. The teacher states his/her problem/issue as briefly as 
possible, using measurable and quantifiable terms to describe the behav,or of practice, if 
DO ble For example, "When I ask the class a question, I find that I almost always 
respond' to the student who raises his hand first. I asked Betty, my Peer Observer team 
member, to come into my class and watch what happened. She noticed that this happens 
almost every time I ask a question of the class. I just can't seem to react fast enough to 
stop myself and what I found out is that I'm calling on the same few students every time. 
Other teachers in the Group may ask clarification questions ONLY, at this time, but are 
not to give any advice or make any judgments. For example, "Does this seem to happen in 
one class more than in others" or "Are these students that raise their hands and that you 
call on the 'better' students in the class?" Avoid questions that really suggest advice or 
make judgments, i.e., "Are you afraid to call upon the "slower" students because they 
might give you the wrong answer?" or, in another instance, "Do you give your students the 
same respect you expect from them?" 
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The Facilitator now asks the other Group members, "What have YOU tried that has 
worked for YOU in a similar situation?" The way this is stated is really a key to the 
success of the group process. Note that the members are being asked to relate something 
they think "should" work for the person requesting the help. (Remember that each of us 
makes a change because we believe what we see or hear may work for US, that is, it 
satisfies a need or desire WE have. Essentially, we can listen to what has worked for 
others and think that perhaps that may work for us, too. The point is: Never give advice 
or make judgments! We really are our own "experts" and the answers lie within us. The 
Group may help us to actualize something within us or help us to have an insight about 
something we already know. This way of expressing ourselves, although somewhat 
awkward, at first, will ensure a judgment-free process and allow for trust, creativity and 
openness in our interactions with the Support Group members.) 

Group members now share experience or solutions that have worked for them. For 
example, "I had a similar situation once in my third grade class. I tried....This really made 
a difference for me." or "Last year when I taught American History, I had some students 
who were always waving their hands before I even finished asking the question. What I 
did was use a deck of student name cards which I would shuffle at the beginning of each 
class. I would then turn over a card and read the student's name. I would then ask the 
question and quickly call upon that student to answer the question. After a while, the 
class got the idea and stopped waving their hands." 

^ This part of the process was the most difficult for the Project teachers to 

implement. Rather than simply state, in a non-judgmental way, something he/she had 
done in the past that was successful, the tendency was to tell the other person what to do. 
Again, during the Friday training session with the teachers this process was focused upon 
and generally teachers were able to implement it as outlined. 

At this point, if no one in the Group has any personal experiences to relate, 
someone in the Group, or the Facilitator, may request a departure from the rules. This 
simply allows the Group to brainstorm some ideas. Everyone in the Group is clear, 
however, that these ideas and techniques may or may not have been tried. Usually, no 
more than three or four minutes should be devoted to brainstorming any one topic. 

The Facilitator now solicits from the Group "what the research" says about this 

particular practice or behavior. This is done so that the member being helped can 

compare what he/she has heard from the other Group members with respect to any 
research-based practices with which the Group may be familiar. 

Specific researched-based classroom practices were presented to the staff of both 
schools during the Friday inservice sessions. These practices became the focus for the 
intervening weeks prior to the next inservice. 

The teacher inservice program included rolc-playing situations wherein members 
practiced interacting with one another in a non-judgmental, non-evaluative manner. 

CLOSURE. Following the problem-solving discussion, the Facilitator asks the 
teacher who requested help what techniques or ideas he/she might try to remedy the 
problem situation, i.e., "The Group has shared some techniques and ideas that have worked 
for them in similar situations (or, "We have brainstormed some ideas.") and you have 
heard what the research has to say. What do you think you might like to try?" The 
teacher makes his/her own conclusions about the actions he/she will take. It may be that 
the teacher will actually choose a few ideas that others have tried successfully and/or are 
research-based practices; it is just as likely that the teacher will select some action(s) that 
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was not specifically presented by the Group, but, rather was "triggered* as a result of the 
Group discussion. 

Once the teacher has stated what he/she intends to try, the Facilitator asks the 
teacher to make a commitment to the Group to report back at the next meeting of the 
Support Group (See: "Review and Feedback"). It is important that the teacher actually 
state what he/she will do and that he/she agrees to report back to the Group. This 
constitutes the verbal contract with the Group. It provides a degree of accountability, as 
well as allows the Group to receive reinforcement that they did, in fact, help a member of 
the Group. 

It was found in the Project support groups that after the group became 
comfortable with the process, there was a tendency to neglect seeking the actual "verbal 
commitment" from the teacher. In almost all cases where this commitment had not been 
made, little attempt at changing the behavior had occurred from one meeting to the next. 

The process now begins all over again with the next item from the prioritized 
agenda. When Groups are in their initial stages of forming, one or perhaps tw 0 problems 
or issues can be handled at a meeting. As Groups mature, however, it is not unusual to 
deal with three or four problems or issues. Regardless of the number of problems 
discussed, it is important that the Group adjourn at the agreed-upon time. 

In the beginning when the Support Groups formed, there was a degree of group 
team-building that was necessary. "How much can I trust this group?" Will they laugh at 
me if I make a mistake?" "Arc they evaluating me as a teacher?" Will I really get any 
help from them?" These are the types of questions and concerns that were typical among 
the Project teachers. Most of these teachers, at least during the first few years of the 
Project, had worked together for some time. This helped in terms of their readiness to 
accept such a program that might open them to some personal vulnerability. 

The inservice training program addressed these issues and worked through a set of 
group activities to help the Project teachers feel more comfortable and productive as 
members of their Group. The expressed purpose of the Support Group is to help teachers 
improve their instructional effectiveness and to accomplish this in a safe and supportive 
environment. Initially, it required much time and effort to build this "safe and 
supportive environment." The effort, however, was deemed worthwhile by the Project 
staff and teachers. The final test of the impact of the Support Group can be examined 
after the Project has terminated and some time has passed. Will the groups still continue 
to operate? Will the teachers continue to find support and professional growth through 
the sessions? 

LOCAL SCHOOL ADMINISTRATORS AND LEADERSHIP DEVELOPMENT 

Each of the two districts in the Project have a superintendent and a principal who 
serve the entire school population. Both leaders are housed in the same building and 
often function as a team in providing leadership to the school. While generally the 
principal is responsible for day-to-day administration of the classrooms and the 
instructional programs, the superintendent is involved in these functions, as well. While 
teachers might usually talk with the principal about instructional matters, it is not 
uncommon for the superintendent to make himself available in these matters. As a result, 
both the superintendent and the principal attended all inservice sessions with their staffs 
and took part equally in the facilitation of the Project goals with teachers. 
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The most critical role of the administrators was that of active and vocal support of 
the aims of the Project and of the teacher participants. Regular monitoring of the 
Coaching process, visitations to classrooms, availability to staff, support of the new 
practices arising from the inservice sessions, providing classroom coverage for the peer 
observation program, facilitating common time for staff to meet these activities proved 
to be the most crucial to the success of the program. 

Monitoring of the Coaching process: Specifically this involved making sure that 
teachers were holding support group meetings regularly; assuring non-interference with 
teacher support group agendas; announcing high expectations to staff regarding the 
frequency of support group meetings; generally making sure that the coaching process 
was, in fact, in place. 

Availability to Staff: This involved responding quickly and efficiently to staff 
needs as pertains the Project* conducting regular and frequent meetings of staff to discuss 
issues surfaced by the Project; periodically attending support group meetings; availability 
to "cover" classrooms for peer observations. 

Support of the New Practices Arising from the Inservice Sessions: Activities 
included participation in all inservice sessions with the teachers, participation at all 
leadership training sessions; assuring that current school policies and procedures are "in 
line" with research practice; providing materials and facilities to support the "new" 
practices; including the new practices as part of the teacher performance appraisal system; 
assuring the present methods of student evaluation and supervisory observation are 
consistent with the practices of the Project. 

Providing Classroom Coverage for Peer Observation Program: Activities included 
personal coverage of teacher classroom, if necessary; providing paid substitutes or parent 
volunteers during peer observation (and/or support group) meeting times; adjusting 
existing schedules to accommodate the Coaching program. 

Facilitating Common Time for Staff to Meet: The most critical activities included 
adjusting existing schedules— recess, lunch, starting and ending times, etc. to accommodate 
peer observations and support group meeting times; calling regular and frequent staff 
meetings to discuss issues arising from the needs of the Project; assisting with teacher 
schedule adjustments— playground duty, itinerant teachers; resolving time conflicts 
between athletics, extra-curricular activities and Project activities. 

Many of these leadership functions arising from the needs of the Project were 
unfamiliar to the administrators at the start of the Project. Some simply arose from 
specific need as the Project progressed while others were known from the beginning. In 
all cases, however, when administrators neglected any of these functions for a length of 
time, concerns arose from the teachers. Often these would be revealed to the 
consultant/trainers during their classroom visitations on the day previous to the inservice 
workshop sessions. Such problems as lack of time for meetings, administrators not 
available to handle problems as they arise, no one available to provide coverage for an 
observation, principal did not attend teacher meeting, use of teacher agenda at support 
meetings to handle "administrative" matters, lack of follow-through with a given program 
or activity resulting from the regular inservice session. More often than not, these 
functions were neglected because of needs from non^participant staff members (high 
school teachers, for example) and the fact that in each school only the principal and the 
superintendent were available to perform such functions. 
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Many of the tasks that were required of the local administrators were usual tasks 
that might be performed by most small school leaders. The Project, however, with its 
emphasis on "change" and its focus on research-based classroom practice, required certain 
knowledge and skills that are not always found in such leaders. In order to assist the 
local leadership, a development program was instituted by the Project consultant staff. 

Beginning the second year of the Project, the two consultants would meet on the 
Thursday prior to the scheduled teacher inservice with the four administrators of the two 
schools. At first, the activities of these sessions dealt primarily with information sharing. 
Sometime during the second year of the Project, it was decided tp structure these meetings 
so that leadership development might be encouraged. 

A leadership development program known as Leadership Mapping Program (LMP) 
was conducted by one of the consultants. At the outset of the Program, each of the 
leaders completed a leadership style instrument, as well as a survey designed to determine 
the degree to which each leader needed to change his behavior specific to the task of 
implementing the NIE Project. A discrepancy analysis was performed to determine 
significant personal and leadership behaviors most likely to assist in successful 
implementation of the Project. The administrators examined their own leadership style 
with respect to a set of research-based indicators, as well. 

The results of this instrumentation provided specific individualized activities for 
each of the administrators in the form of a professional growth plan. The plan was tied 
directly to the leadership requirements of the NIE Project. 

In addition, a needs assessment was completed by each of the administrators to 
determine other areas of concern which might be focused upon at subsequent leadership 
development sessions. Examples of topics selected by the group are: using standardized 
test results for program change, monitoring use of classroom learning time, developing 
observational skills, teacher performance appraisal, and the like. 

A program of time monitoring based on the "time-on-task" literature was performed 
in each building. The building administrators in each respective building were trained to 
use an observational instrument to collect classroom time data. In addition, building-level 
use of time was likewise audited through the use of a survey known as the "Academic 
Efficiency Indicator" which was developed by McREL. 

Other leadership activities included: satisfying individual administrator requests 
for specific resource materials such as professional articles, books and the like; assisting 
leaders to prepare and deliver dissemination programs to other professionals; providing 
opportunities for leaders to attend conferences and other professional activities; 
individual consultation to assist in increasing professional competencies. 
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CHAPTER III 



WHO'S ON FIRST 



This chapter contains a discussion of some of the intermediate outcomes identified 
in Figure 1. The outcomes in terms of changes in student behaviors are reported in 
Chapter IV Originally, the student results were considered to be the most, perhaps the 
ONLY important outcomes of the Follow Through Project. For several reasons, this did 
not turn out to be the case in Cotopaxi/Westcliffe. First, student achievement was good 
in these districts before the project started. Second, student engagement and achievement 
were considered relatively unimportant by the staff in the schools so they selected other 
outcomes as the major focuses for the inscrvice. Third, as illustrated in Figure 1, even if 
there were changes in student behavior, they could not be attributed to the treatment 
unless there were changes in teacher behaviors. 

Unfortunately, as often happens in evaluation and field-based research, there was 
a serious mismafch between the data collected and the outcomes which we wished to 
document. Because no one instructional model was used, no specific teacher behaviors 
were identified which could be observed before and after training. At the request of the 
- evaluator a test of the content was developed by the trainers, but was not administered at 
the end of the inservice sessions. Due to the nature of the peer coaching and support 
groups, any documents created during these processes were considered confidential and 
could not be used for evaluation purposes. The following data were amiable and will be 
reported in this chapter: a school climate questionnaire, interviews with teachers involved 
in the project, a questionnaire administered by NIE, a measure of how time is allocated, 
and checklists completed by observers on instructional techniques and discipline. 

Thane in the sunoort and collegialitv of the te achers for each other and jn the 
ntmosphere of the school. Questionnaires measuring five areas of "school climate" were 
administered in the spring of 1983, in the springs of 1984, and in the spring of 1985 
Respondents answered a total of 80 items in the areas of: student/faculty motivation (15 
items), expectations (19 items), teacher professionalism (20 items), building support of 
teachers (20 items), and parental involvement (6 items). They were asked to rate their 
schools as high, medium, or low on each item. Responses were assigned numeric values 
(high - 3, medium - 2, low » 1) and average scores were computed. Since respondents 
were not identified in any way, responses could not be matched and so no statistical tests 
cuuld be performed. The mean scores are included in Table III.I. Differences in means 
of more than +.50 were noted with asterisks in the table. 
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Table MM 

Sumury of Means Scores on School Climate Questionnaire 



to 



1982-83 



■ ~~ (N=9) 

STUDENT/FACULTY HOTIVATIOM 
1. Special waans exist to recognize and reward students who have 
worked hard, *»He progress, or accomplish something special, 



_School_L 
1983-84 
<N=5) 



2.80 



1984-85 
(N=7) 



2.43 



198~.-83 
<H=9) 



2.00 



2. Instructional techniques that are highly motivating art used 
such as "Team Tournament Games" o r "Concept attainments? 

3. Teachers are able and willing to help each other with motiva- 
tional ideas; staff meetings or other communications arrangements 
are set up to permit this excha nge to go on, . 

4. The building has a low absentee rate compared to other 



1.89 



2.40* 



2.00 



1.78 



2.22 



3.00* 



2,57 



2.56 



11. Teachers voluntarily participate in student activities, 
such as: playing BtuctaU in a game of Softball, operating 
booths at a school ctrnival, helping students prepare food 
for the school picni c, parties, etc, 



2.33 



2.80 



2.29* 



2.33 



12. Teachers make an effort to get to know students and 
their parents on a personal basis. 

13. Teachers will extend their day when requested to do 
so by students 



2.89 



2.80 



3.00 



2.44 



2.89 



2.80 



2.43 



2.56 



14. Teachers provide specific time during theday to meet 
with individual students who need and/or want help with 
academic or personal problems* 



2,78 



2,40 



2.43 



2.11 



15. Teachers encourage students to discuss their problems, 



2.56 



2.43 



2.22 



Overall scale mean 

EXPECTATIONS 



2.49 



1. No one 1 * "written off, 11 



2.78_ 



2.63 



3.00 



2.31 



2.57 



2.38 



2.56 




2 . Some students aren't given easier work. 

3. The same standards are held up to everyone with 
consistency. — 

4. Grouping arrangements are not used to signal to some 
students thev aren't expected to do as we ll as others, 

5. Student academic work is displayed on bulletin boards, 

in halls and in classrooms, etc. 

* indicates relatively large differences^ mean responses 



2.11 



2.40 



2.14 



1.86 



2.11 



2.80* 



2.14* 



1.67 



2.22 



2.20 



2.29 



2.22 



2.78 



2.40 



2.57_ 



2.56 



Sijiooj 2 
1983-84 
(N=6) 



1.33 



1.67 



2.67 



2.00 



2.33 



2.67 



2.17 



1.67 



2.50 



2.05 



2.17 
2.17 



2.00 



2.00 



2.33 



1984-85 
(N=4) 



2.75 



2.25 



3.00 



2.00 




2.25 



2.25 



2.50 



2.00 



2.25 



2.10 



2.25 
1.75 



2.50* 



2.00 



2.50 
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Table ?i: ' (cont.) 
Surowy of Means Scores op School CUmate Questionnaire 



to 
to 



EXPECTATIONS (Cont.) 
6. Student academic projects are featured in the school. 



1982-83 
(N=9) 



1.89 



School 1 
1983-84 
(N=5) 



2.00 



1984-80 
(H=7) 



2.29 



1982-83 
(N=9) 



1.67 



7. long term grouping Is not d one on an ability basis 
6. Teachers responses and support to students they perceive 
as low achievers is no different than their response to 

^rceived hi oh achievers 

9. Teachers flive praise that is realistic and specific, with 
respect to the performance. 



2.22 



2.20 



2.00 



School 2 
1983-84 
(N=6) 



1.33 



1984-85 
(N=4) 



2.00 




Overall scale mean 



TEACHER PROFESSIONALISM 



1. Teachers ere willing to allow other and/or parents to 
visit their classrooms* 

2. Teachers sometimes invite the principal or other 
administrators to visit their cl assrooms, 

3. Teachers are wtU'ng to share successful ideas 



and strategies with each other. 
4. Teachers help each other. 



5. Teachers continue to attend classes/programs 
for individual growth, - 



2.89 



2.56 



2.67 



2.78 



2.67 



2.80 



2.60 



2.80 



00 



2.60 



3.00 



2.43 



2.86 



2.86 



2.43 



2.67 



2.11 



2.22 
2.44 



2.56 



2.67 



2.33 



2.67 
2.67 



2.50 



3.00 



2.75 



2.75 _ 
?-75 



JUL 



29 
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Table Ml.1 (cont.) 
Summary of Means Scores on School Climate Questionnaire 



to 



School 1 



1982-83 



1983-84 



1984-85 



TEACHER PROFESSIONALISM (Cont.) 

6. Teachers demonstrate professional integrity using sick 
leave, professional days, etc. 



(H=9) 



2.09 



(H=5) 



3,00 



(N=7) 



2.71 



7. They cooperate in required meetings (PTA. currte ulum.etc. ) 



2,89 



3.00 



2.57 



8, They demonstrate a positive attitude toward their job, 



3.00 



2.60 



2.71 



9, Teacher j share their suggestions for improvements openly . 

10, Faculty members personally welcome ea ch pew teacher, 

11. Grade level faculty members offer ex tra assistance. 



2.89 



2.20* 



2.43 



2.78 



2.80 



2.29* 



2.63 



HA 



NA 



12. Faculty members invite new teachers to out-of -bui Iding 
luncheons. 



» — ^. . - — . ..^ 

13. Faculty members make sure new teachers are aware 
of the Parent Teacher conference routine, 

14, Compliments are freely given, 



~ — r - " — * — — • — ' — ■ i » — — - ' 

15. Faculty menders enioy each others 1 company. 



. ~ M ... — — M ~- — _ — 

16. Faculty members acknowledge and are proud of the 
accomplishments of other faculty. 



17. Faculty members are aware of each other's interests 
and special abilities. 



~* — — — - • 

18. Faculty menbers seek solutions to differences in a 
positive, open manner. 



c — -■ — — ■ ■ 

19. Faculty members feel teaching is a team effort with 
all contributing their best. 



20. Faculty members share ideas and offer assistance 
to others. 



Overall scale mean 



1,56 



2,22 



2.56 



2.44 



2.67 



2.33 



2.44 



2.67 



?,67 



2.61 



1.50 



2.80* 



2.20 



2.60 



2.60 



2.60 



2.60 



2.80 



2.60 



2.62 



2.00* 



2.14* 



2.14 



2.43 



2.57 



2.29 



2.29 



2.43 



2.43 



2.47 



1982-83 



(H=9) 



2.67 



2.44 



2.44 



2.44 



2.67 



1 .78 



1.67 



2.13 



2.22 



2.67 



2.44 



2.22 



1.67 



2.11 



2.13 



2.29 



School 2 



1983-84 



(N=6) 



2.67 



_2.^3_ 
2.53 
2.50 
2.50 
2.00 



1.83 



2.20 



1.67* 
2.33 



2.50 



2.33 



2.00 



2.33 



2.17 



2,33 



1954-85 



(H=4) 



2.50 
2.50 
2.25 
2.25 
2.75 
2.50* 

2.50* 



1-75 



2.25 
3.00* 



2.75 



3.00* 



2.75* 

2.25 

2.75* 



2.59 



BU1L0IWG SUPPORT TO TEACHERS 

1. Teachers participate on committees for curriculum 
study, and materials selection, etc 
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Table lU.t (cont.) 
Simnary of Means Scores on School CUmate Questionnaire 



School 1 



1982*83 



1983-8A 



1984-85 



BUILDING SUPPORT TO TEACHERS (cont.) 

9. Teachers have a say-so in building wide rules end 
activities that affect them. 



(N=9) 



2.67 



(N=5) 



2.60 



(N=7) 



2,86 



1982-83 
(N=9) 



2.56 



10. Staff time is available to share good ideas that other 
teachers use and to help teachers solve classroom problems. 



1,67 



1.60 



1.86 



2.11 



11. A few, building-wide rules or codes of conduct have 
been developed and everyone supports them and models them. 



2.00 



3.00* 



2.57* 



2.00 



12. Workable Job descriptions exist and/or the principal 
i» willing to discuss and clarify job dutie s and tasks. 



2.00 



3.00* 



13. The principal makes him/herself available to 
discuss teachers' problems. 



2.43* 



2.14 



2.56 



2.40 



2.86 



2.44 



14. The principal is generally aware of problems 
that exist within the building. 



2.22 



2.40 



2,29 



2.44 



15. The principal is effective in relaying teacher con- 



2.33 



2.25 



2.29 



1.85 



School 2 
1983-34 



(N=6) 



2J7_ 



2.00 



2.33 



1,83 



2.83 



2.80 



1.60 



1984-85 
(N=4) 



2.75* 



2.75* 
2.25 



2.00 



3.00* 



2.75 



2.75* 



cerns to the district office and vice versa 

16. Disagreements are openly aired and resolved. 



2.00 



'— a— ~ - — — — ■ * ■ 

U. There is a useable curriculum guide to direct 
Instruction; the guide is used. 



18. Teachers have an opportunity to learn what other teache rs teach, 

19. There have been meetings (or other procedures) for 
securing agreement about curriculum and goals between the 
teacher* in the building and the teachers in the "feeder 
schools" they are connected to. 



1.89 



2.11 



2.00 



20. Teachers who teach the same programs (grade 
level) teach the same objectives. 



2.60 



Overall scale mean 



2.21 



PARENTAL INVOLVEMENT 



2,00 



1.60 



2.00 



1.80 



HA 



2.08 



2.00 



1.50 



1.29* 



2.00 



NA 



HA 



2.20 



1.33 



1.89 



1.67 



2.20 
2.03 



1.67 



2.17 



1.50 

1.75 
1.97 



1.00* 



2.00 



1.75 



2.25 
2.30 



1. Parents are given regular (e.g., weekly) reports on 
student progress and/or notification when students do some- 
thing special. 



2. parents/teacher conferences are held well In advance 

of a decision to give a failing grade or retain a student in a grade. 

3. Suggestions are made to parents about what they can 
do to be Involved in their student's 

learning. 



1.89 



2.44 



2.33 



4. Parents know what the discipline codes of the school are. 



1.89 



5. Parents are contacted when discipline problems 
emerge. 

6. Parents know what is academically expected of 
their child. 



2.67 



2.00 



Overall scale mean 



2.20 



1.83 



2.60 



2.60 



2.20 



2.80 



2.20 



2.37 



^14_ 



2.71 



2.57 



2.29 



2.71 



2.57 



2.50 



1.33 



2.33 



2.00 



2.00 



2.22 



_K78_ 
1.94 



1.83 



2.50 



2.33 
1.83 



2.33 



1.83 
2.11 



2.00* 



2.50 



2.50 
2.00 



2.50 



2.25 
2.29 



34 



The responses to the school climate Questionnair e show vcrv little change which 
can be attributed to the inscrvice program. There was no one area which consistently 
showed improvement for both schools across years. There are several possible reasons for 
the lack of demonstrated effect. It is possible that school climate or at least teacher 
perception of school climate is very difficult to change, and may in no way be related to 
other changes that occur. It may be that there is only a "minimal level" which is needed 
and that this was already present before the project began. It is also possible that the 
instrument used to measure the change in school climate was not sensitive to the changes 
which actually did occur. Some of the technical difficulties with this instrument are 
discussed in Technical Appendix A. Finally, the individuals who responded each year 
were different. The first year all participants and observers returned questionnaires. The 
second and third year only a portion of the participants returned questionnaires. Also, as 
explained in Chapter I, there was a large turnover of personnel in one of the schools in 
the final year. Thus, the responses may not be comparable. 

A second source of information about the change in "school climate" was an open- 
ended questionnaire which was completed by teachers after the final observation A copy 
of this questionnaire is included in Technical Appendix A. Nine teachers completed the 
questionnaires. The responses range from extremely negative to extremely positive. Most 
of the responses were positive, although not excessively so. In response to question F: 
•What did you learn from the inservices?", and question 10: "What do you feel was the 
: *~ best thing about the project?", eight of the teachers included remarks about school climate 
and improved professional relationships with the other teachers. Eight of the teachers 
also mentioned that they felt the support groups and peer observations were helpful, a 
positive outcome of the inscrvice, and that they expected to continue them. 

Changes in instructional strategies or behaviors. Three aspects of teacher behavior 
were examined using a checklist completed by the classroom observers. These were: 1) 
clarity of goals and assignments, 2) clarity of classroom/school rules, and 3) use of 
motivational techniques. Each of these was broken down into specific behav l0 rs which 
the observers indicated were either present or absent on the observation guide (see 
Technical Appendix A) by a check mark. Examples of these behaviors follow: 

Clarity of coals and a ssienments 
The teacher: 

spells out skills/knowledge to be acquired 

identifies major steps in tasks 

lists checkpoints to measure progress 

Clarity of classroom /school rules 
The teacher: 

uses oral reprimands and nonverbal cues 

has a visual list of rules 

deals with present behaviors only 

Use of motivational techniques 
The teacher: 

shows genuine zest and enthusiasm 

identifies the activities that will be fun for 

students 

The number of check marks, indicating that the behavior was observed at least 
once, were summed in each category. The total scores for year 2 were averaged across i the 
eight days of observations. For year 3 they were averaged across two days. The possioic 
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The responses to the school climate qu e stionnaire show verv little change which 
can be attributed to the ir.service program. There was no one area which consistently 
showed improvement for both schools across years. There are several possible reasons for 
the lack of demonstrated effect. It is possible that school climate or at least teacher 
perception of school climate is very difficult to change, and may in r.,o way be related to 
other changes that occur. It may be that there is only a "minimal level" which is needed 
and that this was already present before the project began. It is also possible that the 
instrument used to measure the change in school climat. was not sensitive to the changes 
which actually did occur. Some of the technical difficulties with this instrument are 
discussed in Technical Appendix A. Finally, the individuals who responded each year 
were different. The first year all participants and observers returned questionnaires The 
second and third year only a portion of the participants returned questionnaires. Also, as 
explained in Chapter I, there was a large turnover of personnel in one of the schools in 
the final year. Thus, the responses may not be comparable. 

A second source of information about the change in "school climate" was an open- 
ended questionnaire which was completed by teachers after the final observation. A copy 
of this questionnaire is includes in Technical Appendix A. Nine teachers completed the 
questionnaires. The responses range from extremely negative to extremely positive. Most 
of the responses were positive, although not excessively so. In response to question F: 
"What did you learn from the inservices?", and question 10: "What do you feel was the 
^ best thing about the project?", eight of the teachers included remarks about school climate 
and improved professional relationships with the other teachers. Eight of the teachers 
also mentioned that they felt the support groups and peer observations were helpful, a 
positive outcome of the inservice, and that they expected to continue them. 

Changes in instructional str ategi es or behaviors. Three aspects of teacher behavior 
were examined using a checklist completed by the classroom observers. These were: 1) 
clarity of goals and assignments, 2) clarity of classroom/school rules, and 3) use of 
motivational techniques. Each of these was broken down into specific behaviors which 
the observers indicated were either present or absent on the observation guide (see 
Technical Appendix A) by a check mark. Examples of. these behaviors follow: 

Clarity of goals and assignments 
The teacher: 

spells out skills/knowledge to be acquired 

identifies major steps in tasks 

lists checkpoints to measure progress 

Clarity of classroom /school rules 
The teacher: 

uses oral reprimands and nonverbal cues 

has a visual list of rules 

deals with present behaviors only 

Use of motivation"! techniques 
The teacher: 

shows genuine zest and enthusiasm 

identifies the activities that will be fun for 

students 

The number of check marks, indicating that the behavior was observed at least 
once, were summed in each category. The total scores for year 2 were averaged across the 
eight days of observations. For year 3 they were averaged across two days. The possible 
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scores were: 13 for clarity of rules and assignments, 10 for clarity of school/classroom 
rules, and 7 for use of motivational techniques. These data were collected only for years 
2 and 3. An estimate of the internal consistency (Cronbach's alpha coefficient) of these 
scales was calculated. The results of that analysis are presented in Table III.2. 



TABLE 111.2 

Internal Consistency of Instructional Techniques Scales 

Reliability Coefficient 

1982-83 1983-84 



Clarity of goals/assignments .75 .56 

Clarity of school/classroom .79 .59 
rules 

Use of motivational techniques .90 .71 



The considerable difference in the internal consistency of the scales is troublesome. 
The recording of these behaviors required judgments on the part of the observers and it 
may be that between the second and third year the observers changed their definitions of 
the behaviors they were observing. 

The average scores (across observations) for year 2 and year 3 were compared using 
a t-test for dependent means. The results of this analysis are contained in Table III.3. 



TABLE 111.3 

Changes in Use of Instructional Techniques 

1982-83 1983-84 t p 
Clarity of goals and assignments 

(out of 13) x 9.90 9.80 .25 .805 

SD .78 1.30 

N 10 10 



Use of motivational techniques 
(out of 7) x 4.76 

SD 1.16 
N 10 



4.80 
1.27 
10 



-.10 



.921 



Claritv of classroom and school rules 

(out of 10) x 6.68 6.75 -.28 .789 

SD .97 1.32 

N 10 10 



As can be seen from the above results, there is no evidence that there was an 
increase in the teacher's use of these instructional techniques from vear 2 to vear 3. 
There are several reasons why this result may have occurred. There may have been" a lack 
of consistency in the observers' definition of these behaviors from one time to the next. 
Or the instrument used may not capture the behaviors which changed as a result of the 
workshops. Q_r the majority of the behavior change may have occurred between year 1 
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and year 2 when there was no data to evaluate this change. Qr the teachers may have 
been exhibiting many of these behaviors before training began. It seems to be in this area 
that the trainers and teachers were most dissatisfied with the results of the evaluation. 
Both groups expressed the belief that, in fact, behavior changes had occurred, but that 
our instruments failed to document that change. 

It was for this reason that questions concerning changes in instructional techniques 
and strategies were included in the final questionnaire. All the teachers indicated 
specific instructional techniques which they had tried or were currently using. These 
included: sponge activities, checklists of acceptable behaviors, clarification of class rules, 
cutting transition time, praising or emphasizing acceptable behaviors, asking "higher level" 
questions, memory training techniques, concept attainment, using inquiry, teaching 
inductive thinking, cooperative learning and synectics. Seven of the respondents also 
described the differences in their classrooms since the beginning of their involvement in 
the project. These differences included: being more organized, having better discipline, 
challenging children more, more efficient use of time, being more stimulating, giving less 
paper work, using more whole group instruction, using more inquiry and discovery, using 
classroom meetings, using peer tutoring, and letting children participate in decision- 
making. 

The observers also recorded the frequency with which teachers used different 
classroom management or discipline techniques. Every time the use of one of the ten 
techniques was observed, the observer placed a check mark beside it on the observation 
guide (see Technical Appendix A). These frequencies were averaged across the eight 
observations in year 2 and the two observations in year 3. These data were analyzed in 
two ways. First, the average total frequency of disciplinary actions were compared for 
years 1, 2, and 3. It was anticipated that these would decrease due to better classroom 
management techniques. The results of this analysis are reported in Table III.4. 

TABLE 111.4 

Changes in Total Frequency of Disciplinary Actions 

1981-82 1982-83 1983-84 F p 
x SD x SD x £D : 

total 14.50 6.89 15.03 1.89 9.30 3.28 

freq. 

linear 8.77 .016 

trend 

quadratic 4.54 .06 

trend 

This analysis showed that after a small initial increase in the total number of 
disciplinary actions per day from year 1 to year 2. there was a significant decrease in this 

behavior in year 3. This is an expected change in behavior. 

The second analysis was to determine if the relative frequency (or proportion) of 
appropriate or positive disciplinary techniques increased and the proportion of negative 
or inappropriate disciplinary techniques decreased over the three year period. These 
proportions were analyzed using a MANOVA trend analysis to determine if there was a 
linear or quadratic trend. The results of this analysis are reported in Table III.5. 
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TABLE II 1.5 



Change* in Relative Frequency of Various Disciplinary Technique! 



Behavior 



|| 1981-82 |j 1982-83 \ 



11 x I SD 



Positive behavior* 
teacher stops inappropriate 
behavior immediately 
linear trend 
quadratic trend 
teacher rewards a good 
behavior 

linear trend 
quadratic trend 
teacher purposely move* 
toward and stops near a 
behavior problem 
linear trend 
quadratic trend 
teacher touches a student a 
a form of positive rein* 
forcement 

^linear trend 
quadratic trend 
teacher uses direct, simple 
reprimand 

linear trend 
quadratic trend 

total positive behavior* 
linear trend 



Behavior 



II M 



II -08 



II 



11 

II OS 

II 

II 

II 

II -<7 

II 

II 

II 

II .86 



1981 



Negative Behavior 
teacher leave* students alone 
linear trend 
quadratic trend 
teacher waits to stop or 
ignores inappropriate 
behavior 

linear trend 
quadratic trend 
teacher ignore* a good 
behavior 

linear trend 
quadratic trend 
teacher is impolite to 
students 

linear trend 
quardratic trend 
teacher uses indirect form 
of reprimand 

linear trend 
quadratic trend 



total negative behaviors 
linear trend 



II -05 
II 



|| .OS 
II 



II o 
II 
II 
II 

|| .016 

II 
II 
II 

II 06 

II 

II 

II 

II M 



.15 || .11 
II 
II 
II 

.10 || .11 
II 
II 
II 
II 

.10 || .09 
II 
II 
II 
II 

.04 || .09 
II 
II 
II 

.29 || .36 
II 
II 
II 

.16 || .76 



82 || 1982 

SD II ~x 



.06 



II 

.05 || 
II 



.04 



.08 



0 || .01 
II 
II 
II 

.02 | .010 
II 
II 
II 

.10 || .10 

II 
II 
II 

.16 || .24 



SD 



.06 



.09 



.05 



.05 



.07 



.14 



83 



SD 



.04 



.09 



.02 



.02 



.03 



.14 



1983-84 || F 
SD il 



.11 



.14 



.16 



.04 



.22 



.67 



1983 



.05 



.16 



.09 



.007 



.01 



.32 



.05 | 



1.84 
1.15 



2.05 
.01 



1.42 
6.72 



II 
II 

.08 || 
II 
II 
II 
II 

.05 | 

II S3 
|| 22.39 

II 

.08 | 

|| 6.86 

II 



•16 || 



1' 84 



84 



SD 



.06 || 



.01 
II 01 
II .98 



II 
II 

•XI || 



.08 || 



22.44 
.56 



14.76 
6.59 



•02 || 



II 100 
II 09 



.03 || 
II 
II 
II 



3.56 
12.32 



|l 13 84_ 



, erJc 



*5' 

.Ski. 
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It is difficult to make any general statements about the patterns of positive and 
negative teacher behavior techniques based on the individual behaviors. However, 
overall, the use of positive discipline techniq ues as a category show a definite downward 
trend . Conversely, the use of negative discipline techni nues as a category show a definite 
upward trend. It may be that the training was confusing or contradictory on the subject 
of disciplinary techniques. However, it should be remembered that these proportions 
represent very small frequencies or actual occurrences of behaviors. 

Change in use of time . Several ways in which teachers used time were examined 
for this evaluation. First, the percent of time teachers spent in beginning managerial 
activities, transition, giving assignments, disciplining, working with one or two students 
while others wait, and ending managerial activities were added together to form a scale 
called instructional management time. Second, the percent of time spent on social 
activities, outside interruptions, and other breaks were added together to form a scale 
called non-instructionai use of classroom time. Third, the percent of. time spent on 
lavatory break, recess, lunch, breaks between classes and other out of class time were 
added together to form a scale called scheduled interruptions. Finally, the percent of 
time set aside for reading and math was recorded as allocated time. The percentages of 
time spent in these activities was collected in the classrooms of all of the ten teachers 
involved in the project by trained outside observers. The first dau collection was done in 
the spring of 1982 before any training occurred. Observations were made in each 
" teacher's classroom on three different days and the amount of time spent averaged across 
the three days. During the 1982-83 school year, eight observations were made 
(approximately one a month) and averaged. During the 1983-84 school year, two 
observations (one in September and one in May) were done in each classroom and 
averaged. During the 1983-84 school year, two observations (one in September and one in 
May) were done in each classroom and averaged. These averages were compared using a 
repeated measures ANOVA. The results of the analysis of these variables are reported in 
Table III.6. 
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TABLE 111.6 

Percentage of Time Spent in Various Activities 



Activities 

Instructional 
Management 
beginning managerial 

activities 
transition time 
giving assignments 
disciplining 

working with small group 

while others wait 
ending managerial time 

TOTAL managerial time 



Non-instructional In- 
class Activities 
social activities 
outside interruptions 
other interruptions 

TOTAL 



Scheduled Interruptions 
lavatory break 
lunch 
recess 

breaks between classes 
other interruptions 

TOTAL 

Allocated Time 
reading 

math 



U81-82 


II 


1982-83 


II 


1983- 


84 || 


F 


1 P 


x | SD || 
1 II 


* 1 


SD || 
II 




SD || 
II 






3.76 | 


II 
II 

i.u ii 


1.81 | 


II 

.32 || 


3.09 


II 

1 72 || 
1 II 


13.67 


| .0002 


1.92 | 


II 

.57 || 


4.66 | 


II 

1.35 || 


3.44 


I 1.24 || 


22.72 


| .0001 


1.22 | 


.83 || 


2.32 | 


.97 || 


2.18 


1 1-35 || 


5.75 


| .01 


.24 | 


•35 || 


.24 | 


n<» II 

.23 || 


.40 


1 SO II 

1 II 

1 II 


.013 


1 ?2 
I • io 




II 

43 II 


AO 1 

.Uo | 


II 

it II 

.13 || 


.31 


1 51 II 
1 * 9X ll 


.82 


| .46 


2.52 j 


1 OS II 


1.17 | 


.46 | 
ll 


1.92 


1 -62 | 
1 II 


10.83 


| .0008 


9.80 | 


II 
II 

1 80 II 

II 
II 


10.28 | 
| 
| 


II 

2.01 || 
II 
II 


11.18 


\ II 

1 301 || 
1 II 
1 II 


3.39 


| .056 

1 
1 


I 

1.75 | 


II 
II 

1.48 || 


| 

i.il \ 


II 
II 

1 27 II 
1.Z7 || 


4.83 


1 II 
1 II 
1 3.21 II 


8.95 


1 

| .002 


.25 | 


.26 || 

II 


.39 | 


.u || 


.SO 


1 II 


1.00 


| .39 


.15 | 


.39 | 


.81 | 


1.06 | 


.93 


I 1.27 | 


4.13 


| .03 


2.16 | 


II 

2.02 || 


3.47 || 


II 

2.06 || 


6.25 


1 II 
1 <-30 || 
I 1 


9.32 


| .002 
1 




II 
II 


I 


II 
II 
II 




1 1 
1 1 






1.22 | 


II 

1.90 || 


1.18 | 


.98 || 


2.17 


I 1.41 | 


2.05 


| .16 


16.30 | 


2.53 || 


9.79 | 


1.87 || 


9.42 


| 4.66 


14.09 


| .0002 


11.01 | 


2.87 || 


11.77 | 


171 || 


13.59 


1 2.54 | 


| 6.84 


| .006 


.08 | 


.16 || 


.92 | 


.73 || 


3.84 


I 3.45 | 


| 19.67 


| .0001 


.08 | 


1.11 II 


3.14 | 


1.11 || 


.27 


1 -3° 1 
I 1 


| 39.43 
1 


| .0001 


24.47 | 


II 

3.87 || 
II 


26.80 | 


II 

3.48 | 
1 


29.28 


1 1 
1 1 


| 1.56 
1 


1 24 


no data 


II 
II 


16.23 | 


1 

4.05 | 


16.93 


1 1 
| 6.59 | 


1 

| -.36 


| .72 


no data 


II 


9 69 I 


2 37 1 


1107 


1 3 19 


| -135 





.209 



Several interesting effects of the treatment emerge from this analysis. First, the 
percent of time s pent on instructional management activities as a category increased. The 
increase was statistically significant. This may have reflected a difference in the way 
lessons were presented. Instructional practices presented during training emphasized this. 
It should not be viewed necessarily as wisted time. A measure of the quality of this time 
or how it was actually used is required to judge if this is a positive or negative effect. 

Second, within this category, almost every behavior followed the same pattern. 
There was a large change, either an increase or a decrease, in ever y identified use of time 
after the first vear of training. After the second year of training , a smaller change 
occurred in the opposite direction* i.e. moving back toward the original value. This may 
have been due to the fact that training programs almost always have the largest impact 
after the first year. It may also have been because some observer judgment was required 
in deciding when these activities began and ended. Some inconsistency and inaccuracy in 
making these judgments is to be expected which may have produced some "wobble" in the 
scores. 

Non-instructional use of classroom time increased from year 1 to year 3. However, 
during year 3, only two observations were done and one of these occurred on a day when 
birthdays were celebrated. Thus, the unusually large amount of time spent on social 
activities was due to a non-representative sample of classroom activities. There was some 
indication that the percent of time spent in this category was inc reasing: however, the 
results were inconclusive . 

Time use for scheduled interruptions, as a category, remained rela tively constant 
over the three years. Within this category some interesting results can be observed. 
Lunch time was reduced by the school administration as a result of the training. Recess 
time, which is more directly controlled by the teachers, increased at the same time. The 
percent of time consumed by breaks between classes increased due to scheduling for 
music. However, other interruptions were considerably reduced. 

The percent of time devoted to reading instruction increased. The increase in the 
percent of nmr devoted to math was even larger although the inc rease was not 
statistically significant in either case . 

In summary, the way teachers used the school day appeared to be directly affected 
by the workshops. Most of the changes which occurred were in the direction which one 
would predict given the content of the training. 

Summary. Although we cannot quantify these outcomes, teachers report that they 
have changed the way they teach, "iney use different instructional techniques and have a t 
different focus. Their comments also indicate that they found the peer observations and 
support groups useful and will try to continue with the activities. 
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CHAPTER IV 

WHAT DOES IT ALL MEAN? 



INTRODUCTION 

When the follow-through proposals vere funded in 1980, researchers in education 
were very excited about the notion of engagement rate or time-on task. At last we 
thought we had a student behavior which was quantifiable, relatively easy to observe, and 
seemed to be positively associated with achievement. The four follow-through projects 
included data collection procedures which were intended to provide the information 
necessary to determine if there was a causal link between teacher behavior, engagement 
rate and student achievement. As with most field-based experiments, there were many 
practical considerations which prevented the actual implementation from precisely 
following the research design. Included in this chapter is a summary of the research 
study as it was actually conducted. Those readers who are interested in more detailed 
description of the sample, methodology and instruments, intervention, and other outcomes 
are referred to other chapters in this report. 

DESCRIPTION OF SAMPLE 

Two school districts were involved in this study. Both were small — consisting of 
one school of less than 400 students, K-12-— in rural, mountain communities in Colorado. 
All the teachers who taught grade K-4 in these schools, usually one per grade, were 
involved in the inservice training program. Data are not reported for the kindergarten 
teachers. The principals and superintendents in both districts also attended the inservice 
meetings. After the first year of the Project, one district changed superintendents and 
principal; the other district remained stable until the fourth year of the Project when a 
number of changes took place. The changes are summarized in the following table. 

TABLE IV.l 



CHANGE IN PROJECT PARTICIPANTS 







SCHOOL 1 




II 


SCHOOL 2 




| GRADE 


1 Yrl 


1 Yr2 


| YrS 


|Yr< 


II Yrl 


1 Yr2 


1 Yr3 


1 Yr4 1 


| 1 


1 A 


1 A 


1 A 


1 A 


II B 


I B,C 


1 B 


1 D,B | 


1 3 


1 P 


1 F 


1 F 


1 F 


II G 


1 G 


1 G 


1 H | 


1 3 


| I 


1 I 


| I 


1 I 


II J 


| J 


1 J 


1 J 1 


1 < 


1 K 


1 K 


1 K 


1 K 


II L 


1 M 


1 M 


! N | 


| Principal 


1 0 


1 P 


1 P 


1 P 


II Q 


1 Q 


1 Q 


1 R 1 


1 Sup't 


1 s 


_u 


1 T 


1 T 


II U 


1 u 


1 u 


1 U 1 



'Each letter indicates a different staff member 
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Class sizes were small averaging ai>out 20 students. Up to fifteen children were 
observed in each classroom each year, although there is net always complete data for all 
these children. Children were tracked from the time they entered first grade until they 
were promoted into fifth grade, or the project ended. Children who moved were replaced 
by other child en. The number of children who were observed each year and across years 
is reported in Table IV.2. 



43 



33 



TABLE IV.2 

NUMBER OF STUDENTS OBSERVED 



GRADE 






J 3 




. Jl 


s 




II 


4 




II 

_ II 


Total 


I 
1 


SCHOOL 


11 12 


1 tot 


111 12 


1 tot 


111 


12 


1 tot 


111, 


|2 . 


1 tot 


111. - 


12 


Itot I 


Yrl d»t» 


| 10 1 IS 


|24 


|| 15 | 15 


| SO 


|| IS 


1 14 


| 27 


II 9 


|15 


1 24 


II « 


159 


|105 | 


YrJ d»t» 


j 14 | 14 


1 28 


|| 10 | 15 


1 25 


II 15 


1 is 


| 30 


II 13 


U< 


|27 


II 52 


| 58 


Uio 1 




1- I-- 




II 8 111 


1 19 


II 12 


1 12 


1 24 


II 12 


|9 


121. 


IS2 


132 


1 64 1 


YrJ it Yr2 
YrS d»t» 


| IS | 14 
1" 1- 


| 29 


|| 15 |1S 


1 28 


II is 


U< 


|27 


II 12 


U< 


1 26 


II 56 


| 55 


Uio 1 


Yr2 it Yr3 




||1S | 12 


I 25 


II « 


US 


1 21 


II io 


U< 


1 24 


|| 31 


I 39 


1 70 | 


Yrl - YrS 










II 7 


111 


118 


II 9 


111 


1 20 


II 16 


1 22 . 


1 38 I 


Yr4 d»t» 


| 15 |29' 
1" 1" 




|| 14 111 


1 25 


II H 


Hi 


1 25 


II 15 


1 io 


I 25 


II 58 


1 61 


His 1 


YrS it Yr4 




His |n 


1 24 


II io 


Hi 


1 21 


II 11 


1 io 


121 


II 84 


|32 


|66 | 


Yr2 - Yr4 






II" !" 




II * 


Hi 


I 19 


II 8 


1 9 


1 17 


1! 16 


| 20 


|36 | 


Yrl - Yr4 










II" 


1" 


I-- 


II 7 


1 » 


115 


II 7... 


1 8... 


1 15 J 
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The children in this study were identified as medium to low-income. Their parents 
-generally were employed in agriculture or in businesses supported by agriculture. Many 
of the children had siblings who were included in the study. There were no minority or 
culturally different children in these classes. 

DESCRIPTION OF TREATMENT 

The treatment in this project was inservice provided to the teachers. Due to the 
cost of transporting students to and from school, these districts held school four days a 
week The school day was lengthened so students were in school the same number of 
hours as students who attended five days a week. Thus, teachers were paid to attend a 
day of inservice. Generally, trainers spent the days preceding the inservice in the 
classrooms observing or coaching. 

The inservice had two separate agendas. The first was based on the school 
effectiveness research. The topics included in this area included time on task, allocated 
time peer coaching, group problem solving for teachers, teaching to objectives, 
curri'culum-test congruence, and similar topics. The second emphasis was on exposing 
models of teaching which would enrich the instruction program e.g., mastery learning, 
inquiry, creative teaching. A more detailed description of the treatment is included in 
Chapter II. Demonstrations by experienced teachers were also included in this portion of 
the inservice program. 

The objectives in terms of behavior change were never explicitly stated. Teachers 
where not required to use any specific techniques or activities; rather they were 
encouraged to try new ideas and share the results with the group at the next inservice. 
Although it was initially planned to compare student outcomes for teachers who 
implemented the training to a greater or lesser degree, the small number of teachers and 
the lack of specific objectives made this impractical. 
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DEPENDENT MEASURES 

Two dependent measures were used in this study. One was engagement rate. The 
other was scores on standardized achievement tests. 



Encasement rate was calculated for each observed student in reading and 
arithme^k A trained observer made a visual sweep of the selected children every 10 
minuses during reading and arithmetic instruction. On each sweep, the observer indicated 
wne?ner each cWW wis engaged in the teacher-assigned task or not sc . engaged These 
observations were made on three days during the spring o 1982 on days during 

1982-83 on two days during '983-84, and on two days during 1984-85. The number ot 
time, each child was reported as engaged was divided by the total number of times the 
cMld was observer Thh ratio, which fanges from .00 to 1.00, was defined as engagement 
rate. 

Every spring all the children were given a ^^^^.^j^^^J^i 
the soring of 1982 both schools used the Iowa Test of Basic Skills (ITBS). In 1983 school 
two chTnled to the Stanford Achievement Test (SAT), while school one continued to use 
he ITBS The scores reported to the district included stanines, grade equivalent scores 
nd "rcenSe ranks Normal curve equivalent (NCE) or standarc .core , were n0 
reoorted In an effort to improve the comparability of scores from different tests and 
• ac^ diffe^ grade levels, the percentile ranks were converted » ^<»™^ 
normal distribution). The z-scores can range from -2.32 to +2.32. The z score at tne >u t n 
percentile is .00. 

Another decision regarding comparability was which ^» »^^ bt 'SJ e, on j; 
compare. Because of the great variability in the content of ™™y™^™ h ^JZ!* 
total reading test and total arithmetic test scores were recorded. Using these test cores 
al o improved h probability that most of the content which was taught wou d be 
J pre eTed on the test. If the scores for these subtests was not reported l £ a particular 
student, the achievement test scores were recorded as missing for that student.. 

RESEARCH DESIGN 

Orieinallv the research design included collecting data from a comparison group. 
However fne unique characteristics 8 of the two school districts made finding compara >te 
data unUkely and the cost of collecting such data prohibitive. Therefore this i a ^ sing 

"e.oos test design The weakness of this quasi-experimental design are well- 
f^nS^vSc^ny conclusions drawn from the results of this study must be 
considered tentative. 

The questions which were addressed by this study included: 

1. What were the effects of the inservice on student engagement rate? 

o average engagement rate (ER) of a teacher's class compared to 
average ER for that teacher's classes in previous year s. 

2. What were the effects of the inservice on student achievement? 

o average achievement scores of a teacher's class compared to 
average achievement scores for that teacher's classes in previous 
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year's. 
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o individual student's achievement test scores compared to the 

achievement test scores for that student in previous year's. 

3. What is the relationship between student engagement rate and 
achievement? 

RESULTS 

ENGAGEMENT RATE 

Displayed in Table IV.3 arc the number of students per class for whom engagement 
rate was complete and the means and standard deviations of those scores in reading and 
arithmetic. The same information is presented graphically in Figures IV.l and IV.2. Two 
observations can be made from these data. First, the average scores are quite high and 
the scores are distributed in a negatively skewed pattern. Second, there is a clear patter 
in reading which, although less prominent, is also present in arithmetic. There is a 
general increase from year one to year two, another general increase from year two to 
year three and a general decrease from year three to year four. The scores fall within a 
much smaller range in year two and year three than ir year one or year four. 

Determining what comparisons to make using these dita and the appropriate 
statistical tests for the comparisons depended on the undeilying assumptions we made 
about change in engagement rate. If we assumed that engagement rate depended on each 
individual student's reactions to how the teacher managed the class, then each class of 
children the teacher had from year to year could be viewed as an independent sample. 
This assumes that each child's engagement rate was independent from the other children 
in the same class, that engagement rate is primarily a function of teacher behavior, and 
that children in each successive year received a different "treatment" due to the teacher 
inservice. In this case, the appropriate comparison was between the mean engagement rate 
from year to year and was tested using a t-test for independent samples. Although one- 
was ANOVAs or two-way ANOVAs might have been used, this would have resulted in 
much loss of data due to teachers entering and leaving the project at different points. In 
addition, the unequal numbers and variances were more easily accommodated by t-tests 
that ANOVAs. The results of this analysis are presented in the following table. 
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TABLE IV.4 

INDEPENDENT T-TEST OF CLASS ENGAGEMENT RATE 



I Teacher 




READING 








ARITHMETIC 




| Year 






t 


1 P 


II x r x 2 


1 d t 


1 * 


1 P 1 


1 A 

| 1 vi 2 
| 2 vi 3 
I S vi 4 


| •-.07 
| -.06 
1 4- 07 


1 21 

1 26 
1 27 


.3.04 
-17.82 

s.tKJ 


| <.01 
| <.001 


II --xs 

II -02 
II ni 


1 21 
1 2 ^ 
1 26 


| -4.43 
|. 1.21 
1 53 


| <.001 | 
1 >- 2 ° | 

1 > 2U i 


1 B 

| 1 vi 2 

| 2 V9 3 


| ..14 
1 -.01 


1 22 
! 19 


.3.52 
..80 


I <.01 
I > 20 










t P 

| 1 vi 2 
| 2 vi 3 
j S vi 4 


| ..25 
j -.003 
! .20 


i 21 

| 20 

1 24 


.5.09 
-.19 
18 29 


| <.001 
| >.20 
1 <.001 


II -- 20 
II +- 02 
II 10 


1 21 
| .20 
1 24 


| -3.70 

1 -49 
1 10 64 


1 <oi | 

1 >-20 | 
1 < 001 | 


1 G 

| 1 vi 2 
| 2 vi 3 


| -.06 
1 ..02 


1 27 
1 22 


.2.32 
.1.06 


| <.05 
1 > 20 


II 02 
II ..08 


1 27 
1 22 


1 57 
1 -3 09 


1 >-20 | 
1 <01 1 


| I 

1 1 vi 2 
| 2 vi 3 
1 3 vs 4 


1 ..04 
| -.07 
1 .04 


1 24 

1 22 
1 21 


•1.27 
.2.81 
1.78 


1 >.20 
| <.01 
1 <M0 


II ">18 

II -- 01 
|l -.04 


1 26 
| 22 
1 21 


| -3.45 
1 -.22 
1 -.92 


1 >-20 | 
1 > 20 1 


| J 

| 1 vi 2 
J 2 vi 3 
| 3 vs 4 


1 

| ..09 
| .03 


1 24 

1 23 
! 19 


-8.54 
2.00 
| .68 


| <.001 
| <.10 
1 >20 


II - 06 
II --X4 
II 08 


1 26 
1 25 
1 19 


| -1.27 
| -3.67 
1 1 46 


1 >-20 | 
1 <-0X I 
1 < 20 1 


1 K 

| 1 vi 2 
| 2 vi 3 
1 3 vi 4 


1 01 
| -.07 
| .1.28 


1 20 
| 22 
1 24 


I U 
| -2.24 

1 -3.35 


| >.20 
| <.05 
1 <.01 


II * 03 
II - 06 
II 02 


| 20 

1 23 
* 1 24 


| -.73 
| .1.81 
1 42 


1 >-20 | 

1 <-xo I 

1 > 20 | 


1 M 

| 2 vi 3 


| ..07 


1 24 


.3.24 


1 <.01 


II .05 


1 23 


1 -1 30 


1 > 20 1 



*A negative value indicate* an increase from the first year to the second. 



Of the thirteen comparisons of first versus second year of participation in the 
project, eleven showed an increase in the second year. Eight of these were statistically 
significant. Of the fifteen comparisons of second versus third year of participation in the 
project, thirteen showed and increase. Eight of these were statistically significant. Of 
the ten third versus fourth year comparisons, eight showed a decrease in the fourth year 
in average engagement rate. Three of these were statistically significant. The 
information in Table IV.4 also shows that there was a similar pattern in standard 
deviations from year to year. 

An argument could be made that, in fact, the data from year to year were from 
dependent samples. This assumes that the average engagement rate was completely under 
the control of the teacher and resulted from the group's reaction to the teacher's behavior. 
It also assumes that the student groups from one year to the next were basically the same 
and that each student's engagement rate was dependent upon the engagement rate of the 
other students in the classroom. For this comparison a t-test of dependent samples was 
used. This tested whether there was a statistically significant change in the average 



39 

50 



engagement rate among the teachers from one year to the next. The results of this 
analysis are presented In Table IV, 5. 

TABLE IV.5 

DEPENDENT T-TEST OF AVERAGE ENGAGEMENT RATE 





1 N 
1 


IX, 


1*2 


1 d 


I'd 


1 * • 


1 P 1 
1 1 


| 1982 v» 1982-83 
| READING 
| ARITHMETIC 


1 

'|7 
16 


| .80 

I .79 


| M 
| 89 


| VlO 
1 -.10 


| .08 
1 .10 


| -3.11 
t -2.44 


I <-05 | 
| <.10 1 


| 1982-83 v» 83-84 
| READING 
| ARITHMETIC 


1 

17 


1 91 

( .89 


| .95 

1 .93 


| -.05 
1 -.06 


| .03 

1 .06 


| -3.96 
| -1.84 


1 <oi | 

1 <.20 1 


| 1983-84 v» 84-85 
| READING 
1 ARITHMETIC 


1 

15 


| .95 
f .92 


| .88 
| .89 


1 07 
1 .03 


| .08 
1 .06 


| 1.91 
1 1.21 


| <.20 | 
1 >.20 1 



J 



The negative value indicates that Xj is larger that 



As can be seen from these comparisons, the largest changes occurred during 
the first year of treatment. The increase in reading was statistically significant 
and the increase in arithmetic approached significance. Although the average 
engagement rate continued to increase after the second year, only the increase in 
reading was statistically significant. During the third year of treatment, the 
average engagement rate decreased although this decrease was not as large as the 
increase of the previous two years. 

A third way to examine these data depended on the assumption that 
engagement rate is a function of student characteristics, such as interest, ability, 
previous experience, and development level. If this were the case, then the 
appropriate analysis was at the student level. One such analysis was a comparison 
of student engagement rate from year to year using a t-test for dependent samples. 
Because there was an expectation that older children would have higher 
engagement rates than younger children, the analysis was done for each grade 
level. The results of these analysis are presented in Table IV.6. They are also 
represented graphically in Figure IV.3 and IV.4. 
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TABLE IV.6 

DEPENDENT T-TEST OF STUDENT ENGAGEMENT RATE 





1 V 

1 ™ 


- 

1 Y 1 


- 
y 2 


1 rT 1 


S d 




I n 1 
1 " 1 


| Grade 1*2 
















| Yrl-Yr2 


1 18 


| .80 


.91 


1 *-.12 


.10 


| -5.01 


| <.00l I 


| Yr2-YrS 


1 * 7 


| .90 


.91 


| -.01 


.07 


| -.59 


1 >- 20 1 


| YrS-YM 


1 15 


j .95 


.74 


1 .21 


.14 


1 5.70 


1 <.001 1 


| Grade 2-3 
















| Yrl-Yr2 


| 20 


| .76 


.87 


| -.10 


.18 


| -2.62 


1 <- 02 1 


| Yr2-Yr3 


1 16 


| .93 


| .96 


| -.03 


.06 


| -2.34 


1 < os 1 


1 YrS-Yr4 


1 15 


| .91 


I .91 


1 -.00 


.13 


1 -.10 


t >.20 | 


| Grade 3-4 
















| Yrl-Yr2 


| 20 


| .77 


.90 


1 -.13 


.19 


| -3.09 


1 <- 01 1 


| Yr2-Yr3 


1 19 


| .88 


| .97 


| -.09 


.09 


| -4.23 


| <.001 | 


I Yr3-Yr4 


1 is 


1 .95 


1 .91 


1 .04 


.07 


1 2.42 


t <.05 1 


1 ARITHMETIC ! 1 1 1 1 


| Grade 1 -2 
















| Yrl - Yr2 


1 13 ' 


1 71 


| .92 


| -.21 


.22 


| -3.40 


1 <* 01 1 


| Yr2-Yr3 


1 I 7 


| .96 


| .93 


| .03 


.11 


| .93 


1 >- 20 1 


| Yr3-Yr4 


1 14 


1 .96 


1 .82 


1 .13 


19 


1 2.60 


1 <.05 | 


| Grade 2-3 
















| Yrl-Yr2 


1 21 


| .83 


| .82 


| .01 


| .20 


| .18 


1 >- 20 1 


| Yr2-Yr3 


1 1* 


1 * 91 


1 M 


| -.02 


1 12 


1 -- 77 


1 >- 20 1 


1 Yr3-Yr4 


I 15 


1 .94 


| .86 


| .08 


1 15 


| 2.07 


1 <.10 | 


| Grade 3 - 4 
















| Yrl - Yr2 


1 21 


| .67 


| .88 


| -.21 


1 15 


| -6.42 


| <.001 I 


| Yr2-Yr3 


| 22 


| .80 


1 -94 


| -.14 


1 15 


| -4.55 


1 <.ooi l 


1 Yr3-Yr4 


1 *5 


1 .88 


1 .91 


1 -.03 


1 .17 


1 -.78 


1 >20 1 



*A negative value indicate* that y« it larger y 
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Three interesting results were revealed by these analyses. First, there was 
no substantiation of the commonly heid belief thai children's engagement increases 
as they get older. If this had been the case, there would have been a consistent 
pattern of increasing engagement rate from grade to grade in the same year and 
across time. This was not the case. 

Second, the same pattern of change that was observed at the classroom level 
was observed at the student level. There was generally a large, statistically 
sienificant, increase in student's engagement from year 1 to year 2. There was a 
smaller increase from year 2 to year 3 which was statistically significant in three 
out of the six classes. There was a decrease in student engagement from year 3 to 
year 4 which was statistically significant five out of six times. These changes were 
more consistent in reading than in arithmetic. 

Finally, the average student engagement rate was generally quite high. 

Initially, it was planned to divide the entire sample into groups based on their 
orevious year's achievement test scores to determine if there were different effects 
depending on a child's quartile rank. However, as will be apparent from the achievement 
test results reported later in the chapter, very small numbers of children scored below the 
first quartile making this analysis of little value. Instead, we examined the differential 
effects of length of time in the project. We were looking for patterns in change in 
-engagement rate over time. These data were analyzed using t-test for dependent samples. 
The results of these analysis t-test fo dependent samples. The results of these analyses 
are reported in. Table IV.7. They art presented graphically in Figure IV.5 and IV.6. The 
patterns are 

TABLE IV.7 

DEPENDENT T-TESTS OF STUDENT LEVEL ENGAGEMENT BY TIME 



READING 



ARITHMETIC 

~3\ 



•5TI 



| Group A 
1 Yrl 


.76 | 
.89 1 


31 


-.13 


.17 


,1 
1 

.1.23 


.Ji- 
ll 


1 
1 

.69 | 
.89 1 


SO j 


-.20 


.18 | 


1 

.F.04 j 


<.001 


| Yr2 
| Group B 
| Yr2 


.89 | 
.95 1 


12 


-.05 J 


.07 j 


-2.7S 


II 

-1 


1 

85 | 

.91 1 


16 j 


-.10 J 


..15 J 


1 

-2.75 j 


<.02 


| YrS 
| Group C 
| Yr3 


1 *9* 1 
.80 1 


23 


.15 J 


.17 J 


4.36 


11 

!<.ooij| 


1 

.92 | 
.86 1 


22 j 


..07 [ 


.21 J 


1.46 


<.20 


1 Yr * 

| Group D 

1 Yrl 
j Yr2 


1 - 80 1 
1 * 80 1 
| .97 1 


I 7 1 


-.09 J 
-.08 


.18 j 
.10 


•2.04 
-3.59 


1 II 

Hi 


1 

.84| 
.81| 
.96 1 


16 | 


.04 j 
-.16 


.21 
.14 


.,0 

-4.49 


1 

>.20 


| Yr3 
| Group E 
| Yr2 
| YrS 
1 Yr4 


I -92 1 
1 -92 | 
| .95 1 


13 | 


-.00 J 
-.01 


.07 j 
.12 


-.12 
-.38 


1 11 

!>.*>« 

1 J 20 11 
1 II 


1 

.96 | 
.91 » 
.88 1 


" 1 


.03 j 
.05 


.12 
.15 


.93 
1.16 


1 

>.20 
>.20j 


| Group F 

1 Yrl 
| Yr2 
| YrS 
| Yr4 


1 « 79 1 
I -91 | 
1 -93 | 

| .88 I 


10 


-.12 
-.02 
.05 


.08 
.06 
.06 


-4.59 
-1.29 
2.62 


1 ' II 

1 11 


1 

.74| 
.92 | 
.92 j 
.86 1 


9 | 


..18 
.00 
.06 


.17 
.10 
.13 


-3.13 
.07 
1.42 


1 1 

1 >.20 

1 <.20 
1 1 
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much more readily apparent in the graphs. Groups A, B and C were each in the project 
for two years only. Groups A and B showed increases in engagement rate in both reading 
and arithmetic. Groups C showed decreased in both reading and arithmetic. Groups D 
and E were in the project for three years. These show increases in reading across the 
three years and mixed patterns in arithmetic. Group F were in the project for four years. 
The pattern of change in engagement for this group is similar in reading and arithmetic 
and it reflects the pattern bound at the class level. Because there were only ten students 
in Group F, their engagement rates were graphed (Figure IV.7 and IV.8). These graphs 
make some important points more visible. First, only one student in reading and one in 
arithmetic demonstrated a decrease in engagement rate from year 1 to year 2. From year 
2 to year 3, four students in reading and three students in arithmetic decreased 
engagement rate. In year 3, half of the students were reported to have 100% engagement 
rate. They had reached the top of the scale; they can only go down or stay the same. 
This may partially explain the decrease in engagement which occurred form year 3 to 
year 4. 

Achievement Test Scores 

Displayed in Table IV.3 (page 37) are the number of students per class per year, 
the mean Z-score and standard deviation of the Z-scores from those students in reading 
and arithmetic. These same data are displayed graphically in Figures IV.9 and IV.10. 
The same analyses that were reported for the engagement rate were completed for these 
scores for the same reasons. The rationale will no* be repeated here, it should be 
emphasized that from the beginning of the project, the average class Z-scores were above 
.00 (50th percentile) and the standard deviations were less that 1.00. This makes this 
population unusual and violates the assumption that low-income children necessarily need 
to improve their achievement test scores. 

The results of the analysis of the differences in class means from year to year 
using a t-test of independent samples are presented in Table IV.8. The results 
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TABLE IV.8 



INDEPENDENT T-TEST OF CLASS Z-SCORES 

ARITHMETIC 




I A 

| lvi2 
| 2 vi 3 
3 V3 4 

1 B 

| 1 vt 2 

2 vi 3 
I F 

| lv»2 
| 2 vt 3 

3 v» 4 

G 

J Xv. 2 
| 2 vi 3 



.39 
-.30 


1 26 
\ 28 


| 1.30 
| -1.00 


| >.20 
1 >.20 


.40 
-.23 


1 

1 26 
| 21 


1 

| 1.76 
1 -.67 


1 

| <.10 
| >.20 


-.57 


1 

1 w 


1 

| -2.05 


1 

| <.io 




of the t-tcst 



for dependent samples are presented in Table IV.9. The results of 

TABLE IV.9 

DEPENDENT T-TEST OF CLASS Z-SCORES 



i — — 


1 N| 




*2| 








Pi 


| 1982 vi 1982-83 
| Reading 


| 1 

1 61 

J 6! 


1 

.25 1 


1 

.39 | 
.38 1 


I 

■07| 

-.111 


1 

.39 | 
.23 1 


1 

•41| 
-.55 1 


1 

>.20 | 
>.20 I 


| Arithr«*tic 

| 1982-83 vi 1983-84 

| Reading 


| 1 

1 7 I 

| 7 1 


1 

.38 | 
.29 1 


1 

.49| 
.27 1 


1 

-.10 | 
.01 1 


1 

.32 | 
.11 1 


1 

-.84| 

.10 ! 


1 

>.20| 

>.20 1 


I Arithmetic 

| 1983-84 vi 1984-85 
| Reading 

| Arithmetic 


| 1 

1 «l 

| S 1 


1 

.46| 
.23 1 . 


1 

.63 | 
.51 1 


1 

-.28 1 


1 

.27 | 

.06 1 


1 

-1.26 | 
-2 52 1 


1 

>.20 | 
< 10 1 
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if there were no intervention, 
and Figures IV.ll and IV.12. 

TABLE IV.10 

DEPENDENT T-TEST OF STUDENT ACHIEVEMENT 




| READING 

| Grade 1 - 2 
| Yrl - Yr2 
| Yr2-Yr3 
Yr3 - Yr4 
| Grade 2 - 3 
| Yrl-Yr2 
| Yr2-Yr3 
| Yr3 - Yr4 



N | 


1 *i 1 


y 2 i 


d 1 




12 


1 

1 •« 


i I 

| .65 


1 1 
| -.11 1 


i 

1 M 


10 


1 •« 


| .83 


1 -« 


1 - 70 


21 


j .69 


| .38 _ 


| .31 


| .44 _ 


32 


1 

1 « 


1 

1 - 07 


1 

1 I 7 


1 

| .63 


19 

09 


1 -70 


1 « 


1 « 


1 « 


1 .85 


| .69 


1 -16 


| .56 



-.44 
-1.86 

3.19 

1.30 
2.49 
.85 



| >.20 
| <.10 

<.01_ 

| >.20 
| <.05 
>.2Q 




io • .h. table which are statistically significant are 
The only four t-test out the 18 in the table * hlcn * d whcn thc first year was 
decreases It should also be noted that these dec rease occur ea toward the 

d p e ar"cX-,y high (.69, .70, .Ttong b J^£j*,/ lt \ also curious that 

\ tjsss: tsss££ isrisr* , da ta „«> and t h0S e wh0 
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_ . r ad, 3 ana grade 4 d,„ (.22). It indicates .ha, <h f ; average Z-scr« is very sensitive 
, 0 differences in which students arc includeo .n tnc i«..,s~ 

,„ Figures .V.,3 - ^^^JU^'^^^^ 
s.uoents across time are graphed. These were y of (hese reso]ts sh 

£5 tKfSS'^ l h rra a r, a, g^„, front year .0 year, over iong.r P.rtods 
of time the average differences are very small. 
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The final question which was examined was whether there was a Positive 
r,l 9 tionshiD between engagement rate and achievement test scores. Each analysis included 

onlv ?ho e students f or whom wc had complctc data ( en S a * ement ratC " d * ch \^™ n l 
^t S co?eO for two consecutive years. It is well established that the best predictor of 
£m™» nt test ™ rTta£« achievement. The correlation of students' achievement in 
two cTsecu tVv years provides a criterion by which to judge the strength , of the 
elationship between engagement rate and achievement test scores ^ 
analyses were done because, although engagement rate appears to be a group level 
phenomenon, achievement scores seem to be dependent on the charactenstics of each 
individual. The results of this analysis appear in Table IV. II. 



TABLE 1V.11 



CORRELATIONS OF ENGAGEMENT RATE AND ACHIEVEMENT 



1 

H 

| Yrl 


1 

1 N 
149 


Reading 
| ER with Z 
| r=.28p<.05 
| r=.46D<.01 


II 

| 7 with Z 
I r=.63 p<.01 


II N 

II S3 
1 


Arithmetic 
| ER with Z 
| r=.ll p>.10 
|| r=.54 D<.01 


| Z with Z 
1 r=.54 p<-01 


J 
J 
1 
1 


| Yr2 
| Yr2 


Im 

I 


| r=.5Sp<.01 

| r=.07t».10 


1 r=.69p<-01 
I 


H 53 

1 


| r= 47 p<.01 
|| r= 35 o<.05 


1 r=.90 p<.01 


1 

J 


| Yr3 
| Yr3 


1 


1 r=.10p> ; 01 
| r=.12p>.10 


j r=.53p<.01 


II 43 

1 


| r=.09 p>.10 
|| r=.37D<02 


1 r=63p<01 


1 
1 


1 Yr4 
♦ER = 


Engagement Rate 






•z - 


Achievement Teit Z- 


-scorer. 





Several observations were made about these relationships. First the correlations 
were somewhat unstable; that is, they varied depending on which students scores were 
Tncluded in the analysis. This is particularly evident in the year 3 corre , l0ns 0 
encasement rate and achievement test Z-scores m arithmetic. The correlation is JJ 1 or 
tudcntT who had W 2 and year 3 data and .09 for students who , tad year 3 nd yea 4 
data Second the largest correlations of engagement rates and achievement test scores 
data. Second, , we « g i arithmetic. This is also the year when the largest 

"cr^st "n engageme t rat occurred Third, the largest correlations of engagement rates 
S Tach" temeJt arTas large as the correlations between achievement test scores from one 
year to the next. 

Because of the small size of the ER with Z-score correlations, scatterplots were 
drawn ?" several 5 the correlations. (See Figures IV.19 - IV 26 in Appendix B.) These 
scatTcrplots were used to examine the possibility that the correlations were tttenutted by 
either curvilinearity or restriction of range. No curvilinearity was observed i> an r 0 the 
^« e rnlots However in the y«f 3 data, it is clear that the engagement rate with Z 
^rcorrela^T^aUenuated due to a ceiling effect (-ed Previously in 

SKS as 1 lO^^ge^ ratVin explaining 

the variations in achievement test scores. 
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CONCLUSIONS 

What Were The Effects Of The Inservice On Students Engagement Rate? 

It is fairlv certain that ytudent engag ement rate did increase during the first vear 
t]^ in^c^Scj S fciiiiES There seems to be no compclHng cxpUnanon 
for this increase o her than the existence of the inservice Assuming that the pattera of 
engagement which appeared in the data is real, then the effects of the inservice program 
on engagement rate were "used up" after the first year. Their may be because teachers 
elt that 8 he r suits of maintaining an extremely high level of engagement was not worth 
he effort it required or other costs associated with it. It may also be because the ^ 
clement rates were «ni,rioi»lv high. Year 3 engagement rates were collected by a 
di fferent observer tha n the other years. Teachers expressed concern *^"*£™*y« 
these observations. Although the other year's engagemen rates are also relative! y high 
this is probably due to the definition of engagement established from the beginning of the 
project. 

What Was The Effect Of The Project On Student Achievement? 

Achievement test scores did vary from year to year during the project. However 
"the variation in most cases was not large enough to be important and there was no 
identifiable patterns in these variations. Therefore, the insr.rv.ee appears to have had np 
observable effect on achievement test scores. 

It must be noted that improving achievement test scores was never a »e8jtimatt 
ooal for this Droject Average achievement scores were consistency above the expected 
mean o 0 e' en before ^project. No concern about raising achievement test scores 
was expressed by the district's superintendents by the teachers by -the : P ar nts of the 
children in these schools, or by the trainers. It may also be that the children learned 
things that were not measured by achievement tests. 

What Was The Relationship Between Engagement Rate And Achievement? 

It does not seem to be the case that raising engagement rate necessarily - raises 
achievement test scores. The increases in engagement rate at the class level are not 
accompanied by increases in average Z-scores. This in not conclusive because the students 
Tor whom there is complete engagement rate data are not the same students for whom 
there is complete achievement test data. 

At the student level there is a positive correlation between achievement Z-scores 
and engagement rate, ranging from weak to moderate in strength. It : u . never £ ong 
however, as the correlation between the present and previous year s achievemen t tc s t Z 
scores. It appears that some additional qualifiers must be considered when t m n 
the relationship between engagement rats and achievement These migh t »™e »e 
quality of the engagement and the relationship of the curriculum to what is being tested. 

In the end, it seems we can keep 'the children on task, but we can't necessarily 
make them learn better. 
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CHAPTER V 

SPECIAL EFFECTS 



INTRODUCTION 

Included in this chapter are three special analyses which were done specifically for 
the Cotopaxi/Westcliffe Project. First, is a comparison of engagement rate for different 
periods during the day, completed because the Cotopaxi/Westcliffe Schools were on a 
four-day week Second, is an analysis of the stability of engagement rate, completed by 
Greg Camilli, Ph.D. Third, is the distribution of "student activity study", completed by 
Shelly Karp. 

A COMPARISON OF ENGAGEMENT RATE FOR DIFFERENT PERIODS DURING THE 
SCHOOL DAY. 

Because both Cotopaxi and Westcliffe have four day school weeks and longer 
school days, an analysis was planned to determine if this had any effect on engagement 
rate The 'school day was divided into five equal time periods, and average student 
engagement rate was calculated for each period. The averages were compared using a 
-repeated measures ANOVA. The results of that analysis are reported in Table V.I. 



TABLE V.l 

Comparison of Engagement Rates Across Time Period 





|| 1981-82 


|| 1982 


-83 


II 


1983-84 


II 


I 1 




|| x 1 SD 


II * 


| SD 


II x 


1 SD 


II F 


1 P 1 


Period 1 


11*4.52 | 8.75 


|| 90 40 


| 1.89 


!| 95.50 


| 3.56 


|| 12.20 


| .0006 | 


Period 2 


|| 80.96 | 9.27 


|| 88.51 


| 4.59 


|| 93.94 


| 1.55 


|| 13.84 


| .0003 | 


Period 3 


|| 7V.11 | 12.88 


|i 90.35 


| 3.84 


|| 93.89 


| 4.25 


|| 14.92 


| .0002 | 


Period 4 


|| 86.00 | 10.66 


|| 87.94 


| 5.51 


|| 93.78 


| 4.69 


II 3.26 


| .06 | 


Period 5 


||87.48 | S.20 


|| 91.19 


| 3.42 


|| 94.61 


| 5.58 


II < « 


| .03 | 


F 
P 


|| 3.06 
II .03 


II I-** 
II -28 




II 
II 


.28 

.89 


II 

II . 





The only difference among the engagement rates for the five daily periods 
occurred during year I. However, this disappeared as engagement rates increased. There 
is no evidence that a four dav school week or l o nger school davs had a nepative effect on 
engagement rates. The trend of increasing engagement rates from year 1 to vear 3 was 
consistent across all five da ily periods. 

AN INVESTIGATION OF THE STABILITY OF TEACHER ENGAGEMENT RATES 

The purpose of this research was to establish how stable teacher engagement rates 
are over the course of a school year. This comprises two subordinate questions. First, 
how stable is the rate of an individual teacher? And second, how stable are teachers 
relative to one another? The second question is one that is addressed by classical 
reliability theory in terms of the formulation "how stable is the rank order of teachers 
over occasions." Teacher stability is an important prerequisite for discovering teacner- 
student interactions, or process-product relationships. Obviously, if measured teacner 
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behaviors are highly irregular, then there is little likelihood of these behaviors correlating 
to any significant degree with student outcomes. 

If teachers are to be represented with any average ^engagement rate" scores these 
scores must be relatively consistent over time (Ragosa, Flodcn & Willett, 984) Otherwise 
aJe«« score is not adequately interpretable. Inconsistency may ake the form of 
upwar o downward^ ends over time. Thus, a model for teacher stability m»« be able 
upwaro 01 au u a nriaota bilitv of teachers to classroom circumstances (McGaw, 

to """J* must be formulated as to the important 

7^a£^^S^1^%^ example, subject area is an obviously important 
fac but other iSortant, less obvious facts exist. Instances of the latter are occasion 
lacet, ow oiwBT v . proximity to holidays, and the like. 

™ £s'oovio„? ".«. I, be rhTorde'rins of subjects, and the length of time each ,s 
taught, during a particular day. 

Tf individual teachers show consistent behaviors over time which maintain an 
absolute level ( nat is the absence of time trends), then the individual differences among 
£S ! !2? be stable over time. This means that in general, a given teacher wilt 
S SlSs rank ta o?d C er 0 a V m r ong all teachers Com 

oossible an individual teacher may show consistency, yet his rank order may - vary 
drastically acros occasions. It is also possible that teachers may all show ■consistent, but 
~mw?i!££:*d yet the rank ordering remains relatively constant Therefore, both 
questions are important to examine, and both give unique types of information. 

In this report tentative answers are provided to these two questions; however 

££jra aTat^of as. a fszjs 

ftnany the data were not collected in a fashion that permitted a powerful analysis. 
Several reasons contributed to this and are elaborated next. 

The data were collected from each of eight teachers on eight different occasions 
durine he school year However, the occasions differed across teachers. For example, the 
S IttT or^acher rna, have occ, ,rred in 

minu, ° for the duration of a class. Each child was noted a s e. h er en g age, .h« s 
Sto^ 

and sometimes school-wide testing. 

The result of these nuisance sources of variation are likely to increase the amount 
of error variance in teacher stability estimates. In this case the estimates of s ab hty tha 
a re der ved are likely to underestimate the true stability. However, if the results of his 
study are applied in a manner that is similar to that in which the data were col cct ed 
then the estimates of stability are indeed lower. When sources of error are purposefully 
ignored, the sources are legitimately incorporated into error variance. 
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METHOD 

Engagement rate data were collected from 8 teachers on each of 8 occasion s during 

Reading/Math, the sweep was ignored.) 

Fnr hnth gxg matrices the main entry was the average percentage engaged, across 

particular occasion, and one tcach " ^ as Z^ 1 ^ J duc t0 unavailable data and the' 

^nd following occasions was substituted. 

The actual analysis included two separate strategies. First, a tradit tonal reliability 
analysis was done. In this case, an analysis of variance is performed on the eacher by 
occ^ ons^^trix and the variance components of ^^lZt^t£ & 
EST ^ow^these Sffi^^^ under three different assumptions: 

1. Teachers and occasions are fixed. 

2. Teachers are fixed, occasions are sampled randomly. 

3. Teachers are random, occasions are random. 

Three sets of assumptions limit generalizations about the rank order of " the 
richer Under the first set, statements are strictly limited to these particular teachers 
nt^^ V&r teMni set, an attempt is made to generalize to a broader se of 
occasTons, but restricted to these 8 teacher, In the third set, we attempt to generalize to a 
broader set of both teachers and occasions. 

both reading and mathematics. This information is presented in Table I. 
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TABLE 1 

Teacher Means And Standard Deviations For Reading 
And Mathematics (Standard Deviations la Parentheses) 



Teacher 
1 

r 

3 
4 

5 
6 
7 
8 



Grade 

2 
1 
2 
1 
4 
3 
4 
3 



Reading 


Mathematics 


.913 (.071) 


.893 (.122) 


.892 (.102) 


.891 (.071) 


.921 (.087) 


.939 (-081) 


.861 (.100) 


.969 (.048) 


.929 (.073) 


.867 (.119) 


.907 (.069) 


.844 (-093) 


.886 (.082) 


.889 (.098) 


.845 (.135) 


.791 (.135) 



♦In this class, different teachers taught reading and mathematics, 
the same teacher taught both subjects. 



In every other class, 



In Figure 1 below, the average engagement rates for each teacher are plotted I for 



Figure 1. 
.970 
.950 



Average engagement rates plotted by teacher for reading and mathematics. 



.900 



.850 



.800 
.790 
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Precedina ,h« reliability analyses for engagement '"^^J^ill 
gencralizabili.y-wi.hin subjec., .he ^e'ver the engagemen, ra.es 

occasion. Ra.her than pl«l *« *%2£.*nZ'* a w'eigh.ed linear 

SSTn^SfSS fo^^p.^helv^airo^harp^^orotcasioV, an, inversely by 
The s.anda" P devTa.ion across sweeps. The actual weigh, used was: 

W - N/(S + .050). 

An additive factor was added to the t^™^^^^^ 
displayed in Figure 2 and those for mathematics are given in Figure 3. 

Several comments can be made "^t^^^ 1 ?!^ H^ever'it is • 
there appears to be slightly ™rc ^ency . J ». hem tcnd t0 rcmain 

interesting that in both cases, that the .^"V^i/thViowest engagement rates tend 
constant across the occasions or to C °?* r * ^Lions are roughly one month 

^o increase across the ^^t occasions. "?\*«£.™™£ t l note that the teachers 
apart, so the trend is across the school y;"'.^\ T ^ c ^ ing inscrv ice training over the 
whose engagement rates are measured herein w " c rC " 1 j n " 8 n d cd t0 incrcas e engagement 
course of the school year. This training was difference exists 

rates. The final observation on these in the graphs, is less 

in the average rates for the teachcrs^hc range though ««mc ^ 
than .20 from the minimum rate of about 79 the »J»»J» 0 about 84 t0 . 94) . 
range (deleting the maximum and minimum) is less than .iu kirom 

, . . Ate Bn{ , osa et a i (1984) recommend the assessment of stability by an 
For trend plots, Ragosa et ai. \ivo«) • oHvnntaees over the correlational 

-index of tracking." This index "P; 0Vld « '7 ' ^ ^ 0 ws assessment of 

analyses common in research on teaching F "f^ 0 *™^ index of tracking 
individual differences over more ™*™*£ x t m e paths and thus is applicable 
incorporates explicit statistical models for ndivioua i v recommend thc use 

when time trends in behavior are present. .V 1 **? 1 ™^**?^ -probability that two 
of the Foulkes-Davis G^mma statistic which ^^ ZtrvzV For the 

randomly chosen time paths do not intersect iw } n **^ l \™™^™ giv en by one 

present rending and math ^^X^SXS^ n he Urn paths to thecal 
(1.) minus the ratio of the number of ob served ,nt f ctl ° n ^ utcd bcloWi is onc (1.) the 

SSS roller S?-". specified interval of time. 

For eight linear time paths, the potential number of intersect 
number^ unique ways that eight object, ]^J^™£Z£$£ tV am ma as 1 - 
In Figure 2, 18 actual intersections »« J^™ d * v _^ V^J? « 82 . i t is clear from this 
18/28 = .36. For mathematics, gamma is estima ted .ul 5/28 mat hematics. 
analysis that a more stable rank orde *of i .ehe J « f J "J^^ 0M must conclude 
Moreover, if gamma is interpreted as a probaDility, wen w • time 

that the probability of any two '^^^^ t i ^^T^t ^ range of 
interval (eight months) is ^^J°^\ w obscrvC high 

engagement rates is very small, and it is c oojequ y ma coe fficients are 
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The basic information necessary for a reliability study is an analysis of variance 
for the reacher by ^ ^^^--rS f"? «eo"li to oTinlTnJan 
l^e r fr.«n a so^:"o\—r' Analy^'o? variance tables arc given in Tabte 2 
(reading) and Tables 3 (math) below. 

TABLE 2 

Analysis Of Variance Table For The Teachers By 
Occasions Data Matrix For Reading 



SOURCE 

Occasions 
Teachers 
0 by T 



DF 

7 
7 

49 



MEAN SQUARE 

.003995 
.004489 
.004125 



SOURCE 

Occasions 
Teachers 
0 by T 



TABLE 3 

Analysis Of Variance Table For The Teachers By 
Occasions Data Matrix For Mathematics 



DF 

7 
7 

49 



MEAN SQUARE 

.004264 
.022529 
.005745 



A nreliminary comparison of these tables shows that there is much more variance 
among meb in n mathematics engagement rates, and this effect may be d rectly observed 

in Figure i. -f^^^ f ^d^r^ ^^rl'Tcrr^" 

p h r C ese m n C te5 Th%«%c,ia^ity ^cVnclcnts for reading, and three for mathematics. 

The assumptions for each coefficient are also presented. 
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TABLE 4 

Random. 

VARIANCE COMPONENTS 



MEAN SQUARE 



T,0 fixed T fixed, T, O random 

O ramdom 



Teachers (j r - r" 1 " <t if 

Occasions 0' *t> * 0 c *~ o o fe 

T by O <T re <T f c 

RELIABILITY COEFFICIENT = 



READING Q223 , . 0223 * 0 

'J A r 0297 « .0191* -0 191 

$r e 0041 -0041 -0041 



.630 



.414 -414 



MATHEMATICS 0231 * 0 

C rr f*\. -.0458 -0458* 

.0057 



9? „ .797 



.0057 -0057 
.745 -745 



♦Numbers' marked with asterisks are given as the square root of the variance component. 

As evident in the above table, the reliability ^oefficienu 
C are identical. Therefore, the »ain discus on below nvol^s ^ whcn 

conditions A (both factors fue<P « d ^ f^ v ^" «a the coerficients are the highest, 
we do not attempt to generalise beyond " * &! meaning. On the other hand, 

But in this situation, the m"™"^^^^ of teachers and circumstances, 

aeeeptable, and is also similar to its gamma-eoeffrcent (.75 versus .82). 

It must be realized, however, that the coef f icien. f, or edition C i> » 
estimate o7seneralizabili.v I. rests 'pa.onUy untrue, but 

have been randomly sampled from much largo. W«3£ cieil , for math suggests that 
the last oxercise in generalizab.l.ty rs logzeal— the coel to ion (he 
engagement rates may be stable '"*" h P °^° m n wMch no observations were taken 
. ^^^nS^Vaa^ results is very limited. 
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STUDENT ACTIVITY STUDY 

The data collected during the classroom observations enabled a fine grained 
analysis of student activities during those time periods. The following sets of graphs 
depict those activities. They represent one day's observation and therefore may not be 
typical. 

Figure V. 

Reading 

1 What % of the time were students recorded 

as doing the following activities during 
reading instruction across teachers. 

2 What % of the time were students recorded as 

doing the indicated activities during 
reading. 

345 What were differences among teachers in the 

% of time students were recorded as doing 
the indicated activities during reading. 

6 7 8 What % of the time were teachers recorded 

as doing the following activities during 
reading instruction. 

9 What was the average engagement rate and 

the average % of time teacher did the follow- 
ing during reading: 

1 - questioning 

2 - with small group 

3 - giving directions 

4 - presenting to all 

5 - introducing purpose 

6 - monitoring seatwork 

7 - distributing materials 

8 - grading papers 

9 - organizing material 

10 - disciplining 

1 1 - other instruction 

12 - staying in one place 

13 - moving around 

14 - at desk 



10 



What were average engagement rates for 
each teacher when they were doing the 
indicated activities during reading 
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II What was the average ER for each teacher when 

students working with th: teacher and 
independently ami wh *t % of the time were 
students working with the teacher during 
reading. 

79 

' b8 



READING INSTRUCTION 




READING PERIOD-NOT ENGAGED 




Z 

u 

u 

uj 
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READING INSTRUCTION-ENGAGE 




228 



252 340 



375 



MM 



LISTENING 
[ZD READ ALOUD 



405 482 
TEACHERS 

V//A WRITING 
***** READ SILENTLY 
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READING INSTRUCTION-ENGAGE 




READING INSTRUCT-NOT ENG 




READING PERIOD ACTIVITIES 




READING 



QUESTIONING (27.11%) 



WITH SMALL GRP (22 
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ACTIVITIES 




-INSTRUCTION (31.93%) 



OTHER INSTRUCT I (2.41%) 
GRADING PAPERS (3.61%) 



MON 



I TOR SEATVJOR (4.82%) 



PRESENT TO ALL (7.83%) 
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READING PERIOD ACTIVITIES 
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TEACHER ACTIVITY-READING 




Hi AVE ENGAGEMENT ^ * "ME SPENT 



READING-TEACHER ACTIVITY 



-0 

CO 



UJ 
UJ 
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< 

UJ 

> 
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100 
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QUESTIONING 
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READING INSTRUCTION 




1 1 PERCENT W/TEACH 



12, 13 



For Math 

« \Sl2 in reading 



14, 15,16 



- 3,4,5 in reading 



17,18. 19 



= 6,7.8 in reading 



20 



= 9 



1 « monitoring seatwork 

2 = questioning 

3 = giving directions 

4 « introducing purposes 

5 * with small group 

6 * grading papers 

7 » introducing purpose 

8 * distributing materials 

9 « organizing materials 

10 * disciplining 

11 « other instructions 

12 » staying in one place 

13 « moving around 

14 = at desk 
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MATH INSTRUCTION 



wk r n 1 1« ■ 



LISTENING (19.43%) 



NOT ENGAGED (10.70%) 




ANSWERING QUEST (10. 26%) 
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WORKING W PARTN (3.82%) 



OTHER (4.40%) 



USING MANIPULAT (6. 11%) 



OK'D FREE TIME (7.42%) 
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MATH INSTRUCTION 




waI 1 1 hi; «..-. r.;.< 

. SOCIALIZING 1^ 
NO APPARENT ACT (2.58%) 
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MATH INSTRUCTION-ENGAGED 
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MATH INSTRUCTION-ENGAGED 
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MATH PERIOD ACTIVITIES 
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MATH 



"° MONITOR SEATWOR (16.41%) ~ 



QUESTIONING (15.83%) 

I 

113 

ERIC 



IN- 



ACTIVITIES 



NON- INSTRUCT I ON (39.05%) 




OTHER INSTRUCT I (1.56%) 



GRADING PAPERS (6.25%) 



WITH SMALL GRP (9.38%) 



IPRESENT TO ALL (11.72%) 
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MATH PERIOD ACTIVITIES 





TEACHER ACTIVITY-MATH 
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SUMMARY AND RECOMMENDATIONS 

This section of the report is organized around the layered pattern of impact shown 
in Figure 1, page ii of this report. 

Level I-Changes In Knowledge Of Teachers And Administrators 

Information was provided to teachers and administrators. It was presented in 
many forms: video tapes, demonstrations, and lectures. The teachers' comments indicate 
they they felt overwhelmed at times with the amount of information they received, bm 
generally were pleased with the content which was presented. When requested to do so, 
the teachers were able *j identify specific content which had been presented. 

A climate which was conducive for change seemed to be established. The 
principals provided time and coverage to allow peer observations. Teachers reported that 
they participated in these observations and found them to be useful. Support groups were 
established and met regularly. Many of the teachers indicated that they wanted to 
continue these meetings. There appeared to be no negative consequences for trying new 
ideas or techniques. In fact, at least one teacher indicated that there was too much 
-pressure for trying new ideas. 

Level II— Changes In Behaviors Of Teachers And Administrators 

Although there is r >? quantitative evidence of changes in teacher behaviors, the 
questionnaire data indicate that there were changes. Teachers started specific changes 
.hey had made in instructional strategies or methods; they indicated that they now used 
time in different ways; they reported improvements in collegiality and professionalism. 

Level III— Changes In Behaviors Of Students 

Although initial engagement rates were high, there was a moderate increase in 

engagement rate after the first year of training. There was a small increase in 

engagement after the second year of training, and a decrease afte- the third year of 

training. The decrease is probably due to the fact that engagement rates were inflated by 

measurement error during the thir^ data collection (after the second year of training). It 
seems that there may be an optimal engagement rate which is less than 100%. 

Changes in absenteeism did not occur. This is evidently due to the fact that it was 
relatively low from the beginning. A decrease in the total number of disciplinary actions 
did occur although, again, this was not an area of concern. 

L eve l IV— Change In Achievement Test Scores 

There appeared to be no consistent change in achievement test scores which could 
be attributed to the inservice. There was a positive correlation between engagement rate 
and achievement test scores, but it was smaller than the correlation between previous and 
current achievement test scores. 

The lessons learned from this four year school improvement effort suggest 
recommendations for other small rural districts around three different areas, (1) the 
viability of the four-day school week, (2) the appropriateness of the effective schools 
research for improving the instructional program/increasing student achievement in the 
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, u i a a\ th* Heliverv of staff development to small schools in isolated 
S3 -T .V^i ion. Vo ? fS.. -dience, some recommendations art , ind-ded 
concerainrthe research design and the limitations of data electron proccuu.e, ... ...,s 

kind of field-based research. 

VIABILITY OF THE FOUR-DAY SCHOOL WEEK 

... , r *.l r -nnrt the four-dav school week was instituted as an 

helping districts cope with budge, probes n ated to , the tncr« « ,„ 
cariy 19m. .While enctg, costs hav * »^<^^SS are due to the genera. 

for making better use of the limited education dollar contmucs. 

One of the major questions of districts considering this option is whether , h e 
Kh edui?wiU have . "of InSS ^ult^ose tudcnts^n 

traditional schedule. 

. tW o tnt . n0 i; cv regarding the definition of the school day and the 
school fe™ "'Side™ Cr^k Organizational option as a way of helping 
small ruraTschools make better use of available educational resources. 

The Follow Through Research Project focused. primarily on those aspects of the 

^V'^St^oZ-Z T.'^eVse increases 



worthwhile. 



Recommendation: -If it ain't broke, don't fix iff The notion of time-on- 
task/engagement rates should be viewed in a d.agnosU way. f the -ta - 
indicates that engagement rates are in the 75% range or nigne , 
efforts which focus on other necessary conditions for **«J^?*™™~&& for this 
use of the limited time and resources which small rural schools have available 



purpose. 



OTHER OBSERVATIONS CONCERNING THE EFFECTIVE SCHOOLS RESEARCH 
AND SMALL RURAL SCHOOLS 



ERIC 
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Tne Cotopaxi/neaivwiiw -J?':-- ^7 « , H fl larBCr sc hool improvement en on 01 me 
effective school research. This work Paralleled a « r 8" sc "°^ d p . th . formu lation of 
Mid-continent Regional Edncationa Labora tor J hlC ^ CS ^%^osltions provide a 
-nine propositions "^""fi^^m the Cotopaxi/Westcliffe experience 
co^U^-al ST Zl^^^ns of the effective schools 
research. 

Proposition 1: There is a level at which resources must exist to provide a base for an 
effective school. 

This proposition is concerned primarily about physical facilities. The 
Iff^ive'schools research « that t . not the age o^es.gn oM; 



buUdin, that i important to% udent^ learning, but whether it is well 
?» 1 1 and cTred for The condition of the physical facilities 
re rrstrorgVe^geto^the students and -chers about whether or 
not anvone really cares about what goes on in that building, inc 
chools'in Cotopaxi and Westcliffe were viewed with pride, ^e school 
is still the center of community activities. The buildings may be old, 



but they are generally neat and clean. 
Proposition 2: Physical safety and psychological security must exist in an effective 
school. 

This is an area where there is a significant difference between the 
inner-city schools of a Detroit or Oakland and the schools in Cotopaxi 
S We^ST pSy-i«l and for the most pari psycho logical security 
tends simply not to be an issue. It is a given that he school is a sate 
place oT both teachers and students. The climate in rural schools has 
been described in number of studies as that of an extended family. 

Proposition 3: Student achievement will be greater when the system is consistent in its 
purposes, expectations, methods and evaluations. 

At th, more slobal level, Cotopaxi and Westcliffe did well on this one. 
lie 55? images which exist between the school and community 
Laranfee a common core of values and expectations. Administrator 
»d teachers f« hired who hold values that are consistent with those of 
r c'mmunitV At the more specific, operational level 1 the schoo s 1 
not do as well. A written curriculum, K-12, did not exist or was noi 
u"sed W t T staff turnover, the curriculum tended to change from one 
a 'to the next There was not the coherent content focus across he 
curriculum which research suggests is necessary for an effective school. 

Proposition 4: Student achievement will increase when additional time is available for the 
student(s) to master the prescribed knowledge and skills. 

As suggested earlier, 

with learning in urban areas was not present in Cotopaxi/Westclit 
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The four-day week, allowed for moving most of the extra-curricular 
activities to Friday, when school was not in session. 

Proposition 5: Increased student engagement will increase achievement. 

As indicated earlier, the engagement rates were quite high to begin with 
in the two project schools and increases were realized that were 

improvement resources. 

Proposition 6: High student success at daily tasks will occur when student engagement is 
accompanied by effective instructional practice. 

Effective instructional practices, as used here, include but are not 
Hmited to h gh expectations, clearly stated objectives, daily review of 
orTviously learned material, continuous monitoring of student progress 
lZ ™£ Zl*l and external reward, While all of " ™ 
quite straightforward and apparently appropriate for schools of any 
?ize a few comments are in order concerning the issue of h gn 
expectations" There is considerable evidence that if teachers hold high 
expec ta ions' for students they are likely to live up to those expectations 
TS S feels that a student is going to have trouble learning he 
orobabfy will Because of the intimate knowledge of students and their 
?:^i: Lc sites, there was the temptation to "write off" a younger 
sibling if the older brother or sister was a trouble maker or a slow 
student. 

Proposition 7: Student achievement is highest in organizational settings that provide a 
maximum opportunity for individual student growth. 

Tutoring and team learning are strategies which research suggests are 
ier "?fec"ve approaches to instruction. Such strategies, whi not in 
evidence at the beginning of the project, were encouraged and activated 
by some of the teacher, Rigid ability-grouping ^™~*>**£ 
been demonstrated to have a negative impact on achievement were not 
evidenced in the two schools. 
Proposition 8: Improvement occurs when the values or culture of the organization reflect 
fhe beliei ? that everyone can improve and that improvement ,s expected. 

This proposition and the following— 

Proposition 9: Improvement will occur when the school organization is managed 
encourage and support personal and organizational development. 

relate to the overall capability of an institution to renew itself. As stated at 
the outset of this report, rural education has traditionally not enjoyed th s 
S! capacity. Because of the l £ ° n ° 0 in the 

leadership, orchestrating an ongoing school improvement = gort Ji us not 
cards. This four year effort was a unique experience wh>ch will likely 
replicated in these two schools. 



to 
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PROVIDING STAFF DEVELOPMENT IN SMALL RURAL SCHOOLS 

ji. rontent Two issues appear to be important in determining the content of a 

should be addressed, and S ivcn effective schools research provides one 

sharply focused should the snort oe. ine Cl1 ^ f j schooJ staff 

efficiency issues^ Given the P»'""»" l'~ort the contend of t he staff dcvelopmen, 
SSfif ^^VLXoSJ'Z^^ "than .hose related specific,,,, .0 
increasing engagement rates. 

Thr ouestion of how sharply focused an effort should be must resolve the tension 
which IK,^ ^ practi iLr ^-ey concern is gemng ^ome practtc,. 

^^.^.SrS^ SSSS "no Une between .he two 

extremes and did not satisfy either of the two parties completely. 

Recommendation- In determining the Tocus of staff development/school 
impeov^r/ff She effective schools ^f"^^ ft 

* hose par,icipa,in8 are me '- 

T . nnjverv Creating an ongoing collaborative arrangement between the two 

The isolation w ^ich is inherent ,n runU dis^r cts assignments, had a 

Teachers, who tended to be one of a kind in terms ^oi consultants provided by the 

ass? wisiausr- ^K£?J=^s:'-s£; 

^SSr^Tlhr^U^nece^ 1 ^ S3j3Lf —Tchang. - * 
school program. 

Both the data and the observations of the consultants suggests that the biggest 
impact from the intervention came during the first .year of the P«$ am and the n 1. 
off Schools considering a similar effort may wish to stage the nterve .t. I 
biggest investment of time and -ources comes dun g s wi J sueh 

Sm^'h^^^^ the same with a lesser expenditure of 

tihie and resources. 

The availability of the teachers for a full day of «. prided b, 
the four-day school week, made the "lative long trip to the site. . by the cc ^ 
worthwhile. Using part-time consultants rather than full-time stau Kepi mc v 
within reason for small districts. 
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THE RESEARCH DESIGN/RECOMMENDATIONS REGARDING DATA COLLECTION 
PROCEDURES 

Part of the training was based on the assumption that feedback is important if a 
oerson i to change This feedback should include quantitative data such as infonna ion 
about t me^on-task behavior checklists, and success rate. However, these data cannot be 
viewed as the only legitimate measures of change. Also, if they are to use the data as 
Mtaek teachers ; must find the data credible. Several comments made by the teachers 
^Mv^tt^lS^VlllV of the data. 1) The person doing the data collection 
must oe perceived as competent, i.e. well-trained in the data collection Process ; and 
Srstandtng the teacher's intentions as well as accurately "cord.ng the teacher 
behavior 2) The data collector should be the same person, if possible, each time data is 
collected 3 Every effort should be made to insure that the data are collected on what 
^teacher fSs typical days. 4) Short periods of <^%»J%"^ fhTdlS 
Preferable to long periods on one day. One way to improve the credibility of the data 
coUection is to either train teachers to collect data in their own classrooms or have their 
partners in the peer observation process collect the ('ata. 

A second recommendation concerns the instruments used ^ c ollect the ^ data. 

Untried orocedures or poorly written questionnaires will more than likely produce 
^Wgw«r^Uri«5r« P mcnS which are created or selected should ^vc reliability ^ jnd 
Validity data collected or already available. Often instruments are not sensitiv *™ughxo 

de ect change which does occur, thus masking treatment effects. Also, th « «"™Ren s 

used and their administration should remain consistent over time to produce comparable 

data. 

A final recommendation is that at least minimum anticipated behavior ^hange in 
Mh teachers and students be specified before data col cct.n occur _ s, rum ents sho uld 
beTelected or developed to measure all important outcomes. Baseline data should be 
collected for all important outcomes. 



t 
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TECHNICAL APPENDIX A 



School Climate Questionnaire 

1984-85 Engagement Rate Collection Instrument 

Original Engagement Rate Collection Instrument 

Discipline Checklist 

Instructional Techniques Checklist 

Opcn-Ended Questionnaire 



t 
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SCHOOL CLIMATE QUESTIONNAIRE 



Number of questions: 

Developed by project trainers 

No known validity or reliability data 

Q „,U„: Many of the ,u.s;ions are <^^^Jt^S^SS 
make the questions difficult to answer and the meaning ot tne p 
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listed below are the specific characteristics/behavior, identified for 
iacJ of thff I" .chool cli-af area.. Please rate your .chool 
"-nigh, M-edium, «nd L-low) on each characteri.tic/behavxor . 



STUDENT /FACULTY MOTIVATION 



M L 



2. 
3. 



8. 

V 9. 



Special »aan. exist to recognize and reward "tudents 
who have worked hard, made progress, or accomplished 
something special. 

Instructional techniques that are highly motivating 
•re used such at. "Team Tournament Games" or Concept 
attainment". _ . . 

Teachers are able and willing to help each other with 

_ motivational ideas; staff meetings or other 

communications arrangements are set up to permit ttus 

exchange to tjo on* 

y 4 The building has a low absentee rate compared to 

other schools in the district or state. 

y 5 . There are no broken windows or other evidence of 

" 1 vandalism* 

V/ 6. There is an absence of grafitti in the "strooms. 
~ — 7 . The playground equipment is in good repair and is all 

Serfis'an absence of litter in the building and on 

the playground. 
~. The furnishings are kept in good repair. 

10. The building is well-lighted and clean. 

11. Teachers voluntarily participate in student 
activities, such as: playing students in a game of 
Softball, operating booths at a school carnival, 
helping students prepare food for the school picnic, 

Teacher* £ an effort to get to know student, and 
their parents on a personal, basis. 
Teachers will extend their day when requested to do 

14 :: a cher" a e specific time during the day to meet 
individual students who need and/or want help 
with academic or personal problems. 
15. Teachers encourage students to discuss their 
problems. 

EXPECTATIONS 

1. No one's "written off" 

2. Some students aren't given easier work. 

3. The same standards are held up to everyone-with 
consistency. 



12. 
13. 
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4. Grouping arrangements are not used to signal to 
some students they aren't expected to do as well 
as others. 

5. Student academic work is displayed on bulletin 
, . boards, In halls and in classrooms, etc. 

6. Student academic projects are featured in the 
- - school newspaper /news letter. 

7. Long term grouping is not done on an ability 
basis. 

8. Teachers responses and support to students they 
, perceive as low achievers is no different than 

their response to perceived high achievers. 

-„.■ , 9. Teachers give praise that is realistic and 

_ specific with respect to the performance. 

\y 10. Teachers give immediate praise for the academic 

performance. 

y 11. Rewards for academic performance allows many 
students to raceive recognition. 

y 12. Procedures exist for immediate (daily or weekly) 

recognition of students achievement. 

Low achieving students are not: 

13. Reprimanded more. 

\/ 14. Given easier questions to answer. 

15. Cut off sooner than high achievers. 

16. Given less academic feedback. 

17. Praised either less or more than others. 
~ 18. Treated impolitely^ 

19. Grouped together, 



TEACHER PROFESSIONALISM 



H M L 

^ 1. Teachers are willing to allow other teachers and/or 

parents to visit their classrooms. 

v 2. Teachers sometimes invite the principal or other 

. _ ~~ administrators to visit their classrooms. 

\ 3. Teachers are willing to share successful ideas and 

™~~ ~ ~~ strategies with each other. 

4. Teachers help each other. 

"7 5. Teachers continue to attend classes/programs for 

7 individual growth. 

S* 6. Teachers demonstrate professional integrity using 

— — - gi c x leave, professional days, etc. 

\' 7. They cooperate in required meetings (PTA, curriculum, 

etc. ) 

8. Tfcey demonstrate a positive attitude toward their 
job. 

9. Teachers share their suggestions for improvements 

" ~~~ openly. . 

10. Faculty members personally vlcome each new teacher. 

11. Grade level faculty members offer extra assistance. 
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^ 12. Faculty members invite new teachers to 

out-of -building luncheons . 
\/ 13. Faculty members make sure new teachers are aware of 

the Parent Teacher conference routine. 

____ Compliments are freely given. 

— . — Faculty a^mbers enjoy aach others company* 

— — Faculty members acknowledge and are proud of the 

accomplishments of other faculty. 
17. Faculty members are aware of #ach other 1 * .interests 

and special abilities. 
^ 18. Faculty members meek solutions to differences in a 

positive, open manner. 
m 19- Faculty members feel teaching is a team effort with 

all contributing their best. 
20. Faculty members share ideas and offer assistance to 

others. 



BUILDING SUPPORT TO TEACHERS 



H M L 



K Teachers participate on committees for curriculum 

study, and materials selection, etc. 

^ 2. Teachers contribute suggestions and time to various 

school improvement projects. 

3» Teachers share responsibility for outcomes of school 

projects. 

v " 4* Cross-grade coordination of instruction occurs in the 

school. 

v - 5 • Ample building-level emphasis on academic achievement 

is present. 

2^ 6. Workable arrangements have been made for handling 

discipline activities that require the involvement of 
the principal and other staff. 

JjJ 7. Praise and •strokes" are regularly provided to 

teachers. 

8. Personal concerns of teachers are listened to by the 

principal and other teachers. 

s * y/ 9. Teachers have a say-so in building wide rules and 

activities that affect them. 

10. Staff time is available to share good ideas that 

other teachers use and to help teachers solve 
classroom problems. 

I* • * * ew # building-wide rules or codes of conduct have 

been developed and everyone supports them and models 
them. 

12# Workable job descriptions exist and/or the principal 

is willinq to discuss and clarify job duties and 
tasks. 

^ 13. The principal makes him/her self available to discuss 

s teacher's problems. — 

14. The principal is qenerally aware of problems that 

exist within the building. 
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15. The principal i. effective in relaying teacher 
concern, to the di.trict office and viceSversa. 

16. Disagreements are openly aired 

1?! Were is a uaeable curriculum quide to direct 

' instruction; the guide i» u»ed. 

IS. Teachers have an opportunity to learn what other 
teacher* teach. 

19. There have been .eetings (or other procedure.) for 

: ••curing agreement about curriculum and goal, -between 

"eleachers in the building and the teachers *n the 
-feeder schools' 1 they are connected to. 
\/ 20. Teacher, who teach the »ame programs (grade level) 
teach to the same objective.. 



PARENTAL INVOLVEMENT 



H M L 



Parent, are given regular (e.g., weekly) report, on 
•tudent progre.. and/or notification when .tudente do 

./ 2 ^./te^er^nference. are held well in advance 

^— 2 * of I decision to give •failing grade or retain a 

student In a grade. 

y 3. Suggestions .re made to parent, about what «u y can 

d o to be involved in their student's learning. 

y 4. .Parent, know what the di.cipline codes of the .chool 

v/ 5. Parents are contacted when di.cipline problems 

6 . J«ente know what is academically expected of their 
child. - 
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1984-85 ENGAGEMENT RATE COLLECTION INSTRUMENT 



Developed by project staff 

Index of concordance - .905 

(2 observers, 3 observations, 14 students) 

No known validity data 

There was some question about the observers' ability to make-the judgments 
required in dTstinguishing between instructional activities and instructional 
management activities 
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Original Engagement Rate Collection Instrument 
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ENGAGEMENT RATE DURING INSTRUCTION 



Teacher 
Date 



Time Swee? 3egan 

Ho. of Students Observed 



No, of Students Absent 
Total No, of Students Engaged at 

End of Sweep 
Time Swee? EndetT 




Engagement Rates During Instructional Management or Interruptions 
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JSngageme 



ant rates during INSTRUCTIONAL MANAGEMENT OR INTERRUPTIONS 



Teacher^ 
Date 



T iir.e S we e p Beg a n 

NoTof Students Observed^ 
No. of Students Absent_ 



Total No. of Students Engaged at 

End of Sweep . 

Time Sweep Ended 



| 2 I 3 1 4 | 5 I 6 1 7 I 8| 9 1 10 I 1 1 1 1 2 I 13 1 14 j 15 




walking around room 
standing in o ne place 
sitting at de s 
other ~~~ 




Non-instructional Activities 
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Hon-instructional Activities 

Te ac he r „ 

Date 
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Engagement Rates 
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in 



Ifeadiar 
tote 



Tine Sweep Began 

No # of Students Observed 
No # of Students Absent 



TotaTrto. of Students Engaged At End ui 

Sweep 

Ture Sweep Ended 
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NONACAMIMIC ACTIVITIES 



Dote 







Out of 
Class 


In C 


loss 












Lav Break 


Lunch 


Recess 


Break between classes 


Other 


Beginning Managerial Activity 


Transition time 


Teaclier giving or 
explaining assignment 


Teacher disciplining 


Teacher working with 2 
students — others wait 


Ending managerial activities 


Social Activities v 


Outs ide Interrupt ions 


r- 

r 


iv«an 


T j me 
Ended 


Dor ion 


*D'CO 


c - * «» * 
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Discipline Checklist 
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DISCIPLINE 



SECTION III 

a ) TEACHER 

b) DATE 



c ) SCHOOL 



DIRECTIONS 



: put a check after each behavior each time it occurs. 



1 . Teacher leaves classroom- 
students are present I 


/ 

)/ 

w 




■ 


2. Teacher stops an inappropriate 

behavior at once y 


/ 






3. Teacher waits to »top an inappro- 
priate behavior, ignores it 








4. Teacher rewards a good behavior 








5. Teacher ignores a good behavior 


? 






6. Teacher purposely moves toward 
and stops near a behavior problem 








7. Teacher is impolite to students 








8. Teacher touches a student as form 
of positive reinforcement 




\ :• 


1 t 


9. Teacher uses direct/simple 
reprimand 


v' v 






10. Teacher uses indirect form of 
reprimand 
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+ * ^rk mark next to each behavior each time it occur 
DIRECTIONS: Put a check mar* next w 

a) Clarity of Goals and Assignments 
The Teacher: 



Check Marks 




9) gives clear and specific feedback about questions of 
«rnrients and responses. 

10) praises all students who deserve it. 

often and spends time discussing the 



11) gives directions 

c»ructure of the lesson. 



12) gives explanation in response to student needs, 



13) has a regular routin 



e at the beginning of each class. 



b) Classroom/School Rules 



1, has a workable and consistent routine and policy for 
1> ■ ''f — r^ eies or ruleS - 



0 
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The Teacher: 



Check Marks 



2) .onitors students to see if they understand their 
responsibilities. 



3) holds the students accountable for keeping rules. 



4) »akes sure the consequences for good and bad behavior 
are clearly »^» rstood and consist ently applied. 



5) uses oral repirinands and nonverbal cues. 



6) has a visua 



l list of rules. 



7) is involved with students in a personable and 
f r i e ndlv manner. 



8) deals with present behaviors only. 



9) accepts no excuses from students given for violation 
of a ru le. 

1W avoids use of "language of unacceptance. 1 




1 ) has special means to recognize and reward students 
who have worked hard and made progress. 



2) uses instructional techniques that are highly 
motivating (e.g. . "Team Games") . . 

3) is willing to work with other teachers sharing 
motivational ideas. 



4) shows genuine zest and enthusium. 

5) identifies the activities that will be fun for 

students* 

6) identifies the activities that will be of particular 
<nt-Pllectual challenge for students. ^ 



7) provides learning ac 



tivities of variety and wide 



range of intellectual challenge, 
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OPEN-ENDED QUESTIONNAIRE 



t 
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• T^CEKR IMPRESSIONS CP THE COKPAItlAESICIlPf S KUOW THROUGH PROJECT 
on* or bef ou» March 14, 1965. 

1. Bow long have you been participating in the project? 

2. Indicate the grade level taught. 

3 . Please describe your general impressions of the project. 

4 . Do you feel as though the data collection process accurately reflects your classrooms) 
behavior? Why or why not? 

5 . Do you think these data should have bean obtained differently? How? 



6. Do you feel as though the inservice training was valuable? Why or why not: 



r _ xy, a -jrifipTvicea 7 Pleass be specific. 

7 # What did you learn from the xnservices . rxca 



8 Describe your classroo* now as opposed to 1*81-82. £f IjW* «»t 

2 the Project, indicate any changes in your classroom this year.) 



9 Describe any new teaching strategies and/or behaviors you have tried or are now 
* implementing as a direct result of the project. 

10. What do 'you feel was xhe best thing about the project? 

11. ' What do you feel was the worst thing about the project? 
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